[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:25 dist-test-slave-njxd krb5kdc[21838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:25 dist-test-slave-njxd krb5kdc[21838](info): set up 2 sockets
Aug 09 19:25:25 dist-test-slave-njxd krb5kdc[21838](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 19:25:26.523264 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:25:26 dist-test-slave-njxd krb5kdc[21838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:25:26.648526 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:41177
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:41177
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:26.729723 21852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:26.730003 21852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:26.730041 21852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:26.731220 21852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:26.731289 21852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:26.731314 21852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:26.731348 21852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:26.731372 21852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:26.732928 21852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:41177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:41177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:26.733222 21852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:26.733464 21852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:25:26.738775 21852 server_base.cc:1034] running on GCE node
W20260809 19:25:26.738803 21860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:26.738744 21858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:26.738730 21857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:26.739221 21852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:26.739524 21852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:26.740677 21852 hybrid_clock.cc:648] HybridClock initialized: now 1786303526740672 us; error 34 us; skew 500 ppm
Aug 09 19:25:26 dist-test-slave-njxd krb5kdc[21838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303526, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:26.743091 21852 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:25:26.743546 21852 webserver.cc:460] Webserver started at http://127.21.81.254:45343/ using document root <none> and password file <none>
I20260809 19:25:26.743777 21852 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:26.743844 21852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:26.743984 21852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:26.745070 21852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "f121866b99a24031b35dd2c3762adbe0"
format_stamp: "Formatted at 2026-08-09 19:25:26 on dist-test-slave-njxd"
server_key: "ddd012f627a01ad69f910b579326c845"
server_key_iv: "e691ee9ef2652116b9fd84c234ddfa1b"
server_key_version: "encryptionkey@0"
I20260809 19:25:26.745461 21852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "f121866b99a24031b35dd2c3762adbe0"
format_stamp: "Formatted at 2026-08-09 19:25:26 on dist-test-slave-njxd"
server_key: "ddd012f627a01ad69f910b579326c845"
server_key_iv: "e691ee9ef2652116b9fd84c234ddfa1b"
server_key_version: "encryptionkey@0"
I20260809 19:25:26.746881 21852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:25:26.747586 21868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:26.747751 21852 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:26.747856 21852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "f121866b99a24031b35dd2c3762adbe0"
format_stamp: "Formatted at 2026-08-09 19:25:26 on dist-test-slave-njxd"
server_key: "ddd012f627a01ad69f910b579326c845"
server_key_iv: "e691ee9ef2652116b9fd84c234ddfa1b"
server_key_version: "encryptionkey@0"
I20260809 19:25:26.747933 21852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:26.767453 21852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:26.769258 21852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:26.769424 21852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:26.775437 21852 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:41177
I20260809 19:25:26.775481 21924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:41177 every 8 connection(s)
I20260809 19:25:26.775872 21852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:26.776875 21925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:26.779850 21925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0: Bootstrap starting.
I20260809 19:25:26.781122 21925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:26.781917 21925 log.cc:824] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:26.782821 21925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0: No bootstrap required, opened a new log
I20260809 19:25:26.784391 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 21852
I20260809 19:25:26.784498 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:26.785223 21831 external_mini_cluster.cc:1428] Setting key f7fa38dc0d8a30fcb5bb217db90ce26f
I20260809 19:25:26.786711 21925 raft_consensus.cc:348] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f121866b99a24031b35dd2c3762adbe0" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 41177 } }
I20260809 19:25:26.786901 21925 raft_consensus.cc:374] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:26.786940 21925 raft_consensus.cc:729] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f121866b99a24031b35dd2c3762adbe0, State: Initialized, Role: FOLLOWER
I20260809 19:25:26.787107 21925 consensus_queue.cc:260] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f121866b99a24031b35dd2c3762adbe0" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 41177 } }
I20260809 19:25:26.787204 21925 raft_consensus.cc:388] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:26.787237 21925 raft_consensus.cc:482] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:26.787281 21925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:26.789001 21925 raft_consensus.cc:504] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f121866b99a24031b35dd2c3762adbe0" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 41177 } }
I20260809 19:25:26.789150 21925 leader_election.cc:302] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f121866b99a24031b35dd2c3762adbe0; no voters:
I20260809 19:25:26.789402 21925 leader_election.cc:288] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:26.789426 21930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:26.789578 21930 raft_consensus.cc:686] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [term 1 LEADER]: Becoming Leader. State: Replica: f121866b99a24031b35dd2c3762adbe0, State: Running, Role: LEADER
I20260809 19:25:26.789770 21930 consensus_queue.cc:237] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f121866b99a24031b35dd2c3762adbe0" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 41177 } }
I20260809 19:25:26.789796 21925 sys_catalog.cc:564] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:26.790505 21931 sys_catalog.cc:455] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f121866b99a24031b35dd2c3762adbe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f121866b99a24031b35dd2c3762adbe0" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 41177 } } }
I20260809 19:25:26.790838 21931 sys_catalog.cc:458] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:26.790738 21932 sys_catalog.cc:455] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f121866b99a24031b35dd2c3762adbe0. Latest consensus state: current_term: 1 leader_uuid: "f121866b99a24031b35dd2c3762adbe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f121866b99a24031b35dd2c3762adbe0" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 41177 } } }
I20260809 19:25:26.791371 21932 sys_catalog.cc:458] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:26.791868 21937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:26.792829 21937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:26.798193 21937 catalog_manager.cc:1347] Generated new cluster ID: dd69b910db9747c9bc6394eb4be75340
I20260809 19:25:26.798254 21937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:25:26 dist-test-slave-njxd krb5kdc[21838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:26.806970 21929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:26.786221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46270 (local address 127.21.81.254:41177)
0809 19:25:26.786719 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:26.786737 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:26.786766 (+ 29us) server_negotiation.cc:407] Connection header received
0809 19:25:26.790719 (+ 3953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:26.790727 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:26.790786 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:26.790901 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:26.794039 (+ 3138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:26.795854 (+ 1815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:26.797718 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:26.798520 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:26.798659 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:26.802554 (+ 3895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:26.802558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:26.802558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:26.802577 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:26.804650 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:26.805125 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:26.805133 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:26.805134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:26.805299 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:26.805594 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:26.805595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:26.805595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:26.805849 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:26.806135 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:26.806371 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 19:25:26.806505 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":384,"threads_started":1}
I20260809 19:25:26.812765 21937 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:26.813292 21937 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:26.816223 21937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f121866b99a24031b35dd2c3762adbe0: Generated new TSK 0
I20260809 19:25:26.816414 21937 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:25:26.856392 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:41177
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:26.953207 21952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:26.953442 21952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:26.953470 21952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:26.954806 21952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:26.954850 21952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:26.954878 21952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:25:26.956560 21952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:41177
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:26.956825 21952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:26.957063 21952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:26.963553 21961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:26.963654 21958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:26.963522 21957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:26.963778 21952 server_base.cc:1034] running on GCE node
I20260809 19:25:26.964044 21952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:26.964309 21952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:26.965643 21952 hybrid_clock.cc:648] HybridClock initialized: now 1786303526965636 us; error 36 us; skew 500 ppm
Aug 09 19:25:26 dist-test-slave-njxd krb5kdc[21838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303526, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:26.968652 21952 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:25:26.969149 21952 webserver.cc:460] Webserver started at http://127.21.81.193:38105/ using document root <none> and password file <none>
I20260809 19:25:26.969412 21952 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:26.969484 21952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:26.969666 21952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:26.970892 21952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "a5b387083acc40c4867b9d9485f94ba3"
format_stamp: "Formatted at 2026-08-09 19:25:26 on dist-test-slave-njxd"
server_key: "de4660a7af8812b2e3be406ec847d641"
server_key_iv: "c0bcd53f06cc84729e359cccdbc7aedd"
server_key_version: "encryptionkey@0"
I20260809 19:25:26.971390 21952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "a5b387083acc40c4867b9d9485f94ba3"
format_stamp: "Formatted at 2026-08-09 19:25:26 on dist-test-slave-njxd"
server_key: "de4660a7af8812b2e3be406ec847d641"
server_key_iv: "c0bcd53f06cc84729e359cccdbc7aedd"
server_key_version: "encryptionkey@0"
I20260809 19:25:26.973064 21952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:25:26.973881 21969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:26.974053 21952 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:26.974161 21952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "a5b387083acc40c4867b9d9485f94ba3"
format_stamp: "Formatted at 2026-08-09 19:25:26 on dist-test-slave-njxd"
server_key: "de4660a7af8812b2e3be406ec847d641"
server_key_iv: "c0bcd53f06cc84729e359cccdbc7aedd"
server_key_version: "encryptionkey@0"
I20260809 19:25:26.974244 21952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:26.984614 21952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:26.986841 21952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:26.987061 21952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:26.987394 21952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:26.987774 21952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:26.987830 21952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:26.987871 21952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:26.987895 21952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:26.998574 21952 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:40455
I20260809 19:25:26.998642 22089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:40455 every 8 connection(s)
I20260809 19:25:26.999013 21952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:25:27.003300 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 21952
I20260809 19:25:27.003412 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:25:27.003576 21831 external_mini_cluster.cc:1428] Setting key f46c4a8d85a23898c9946a44e26dfc6b
Aug 09 19:25:27 dist-test-slave-njxd krb5kdc[21838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303526, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:27.010038 21929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:26.999861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:58827 (local address 127.21.81.254:41177)
0809 19:25:26.999923 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:26.999925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:27.000255 (+ 330us) server_negotiation.cc:407] Connection header received
0809 19:25:27.000975 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:27.000976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:27.000991 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:27.001050 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:27.002216 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.002727 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.004066 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.004362 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.004431 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:27.007063 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:27.007065 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:27.007065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:27.007075 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:27.008554 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:27.008987 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:27.008989 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:27.008989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:27.009053 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:27.009326 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:27.009328 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:27.009328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:27.009449 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:27.009527 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:27.009837 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 19:25:27.009942 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:25:27.010234 22092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:26.999827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:41177 (local address 127.21.81.193:58827)
0809 19:25:27.000079 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:27.000125 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:27.000883 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:27.001122 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:27.001125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:27.001938 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:27.002130 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:27.002136 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.002814 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.002815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:27.003961 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:27.003964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.004452 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.004454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:27.004620 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:27.005155 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:27.005166 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:27.006951 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:27.008659 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:27.008668 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:27.008671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:27.008905 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:27.009124 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:27.009125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:27.009125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:27.009248 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:27.009518 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:27.009526 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:27.009700 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 19:25:27.009814 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 19:25:27.009982 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":112,"threads_started":1}
I20260809 19:25:27.010811 22090 heartbeater.cc:347] Connected to a master server at 127.21.81.254:41177
I20260809 19:25:27.010896 22090 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:27.011099 22090 heartbeater.cc:511] Master 127.21.81.254:41177 requested a full tablet report, sending...
I20260809 19:25:27.011716 21886 ts_manager.cc:194] Registered new tserver with Master: a5b387083acc40c4867b9d9485f94ba3 (127.21.81.193:40455)
I20260809 19:25:27.012740 21886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:58827
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:25:27.030524 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:41177
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:27.126725 22095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:27.126950 22095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:27.126976 22095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:27.128126 22095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:27.128163 22095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:27.128189 22095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:25:27.129696 22095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:41177
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:27.129889 22095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:27.130069 22095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:27.135434 22101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:27.135629 22103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:27.135452 22100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:27.135788 22095 server_base.cc:1034] running on GCE node
I20260809 19:25:27.135994 22095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:27.136255 22095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:27.137441 22095 hybrid_clock.cc:648] HybridClock initialized: now 1786303527137422 us; error 31 us; skew 500 ppm
Aug 09 19:25:27 dist-test-slave-njxd krb5kdc[21838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303527, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:27.140064 22095 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:25:27.140555 22095 webserver.cc:460] Webserver started at http://127.21.81.194:38481/ using document root <none> and password file <none>
I20260809 19:25:27.140813 22095 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:27.140883 22095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:27.141034 22095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:27.142149 22095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "ff182406e6774c9e92e2326fae8118c9"
format_stamp: "Formatted at 2026-08-09 19:25:27 on dist-test-slave-njxd"
server_key: "690c48bb04e6a2f5b424d1291cf0f123"
server_key_iv: "59a031c22a5ecbafbb10571789b08b25"
server_key_version: "encryptionkey@0"
I20260809 19:25:27.142570 22095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "ff182406e6774c9e92e2326fae8118c9"
format_stamp: "Formatted at 2026-08-09 19:25:27 on dist-test-slave-njxd"
server_key: "690c48bb04e6a2f5b424d1291cf0f123"
server_key_iv: "59a031c22a5ecbafbb10571789b08b25"
server_key_version: "encryptionkey@0"
I20260809 19:25:27.143960 22095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 19:25:27.144706 22111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:27.144891 22095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:27.144982 22095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "ff182406e6774c9e92e2326fae8118c9"
format_stamp: "Formatted at 2026-08-09 19:25:27 on dist-test-slave-njxd"
server_key: "690c48bb04e6a2f5b424d1291cf0f123"
server_key_iv: "59a031c22a5ecbafbb10571789b08b25"
server_key_version: "encryptionkey@0"
I20260809 19:25:27.145068 22095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:27.155375 22095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:27.157301 22095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:27.157473 22095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:27.157786 22095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:27.158107 22095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:27.158162 22095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:27.158197 22095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:27.158227 22095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:27.168017 22095 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:39919
I20260809 19:25:27.168078 22235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:39919 every 8 connection(s)
I20260809 19:25:27.168433 22095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:25:27 dist-test-slave-njxd krb5kdc[21838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303527, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:27.176139 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 22095
I20260809 19:25:27.176260 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:25:27.176462 21831 external_mini_cluster.cc:1428] Setting key 432662912ecc88df9e0efb0336dadb09
I20260809 19:25:27.178406 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:27.169146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:41177 (local address 127.21.81.194:32787)
0809 19:25:27.169368 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:27.169407 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:27.169997 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:27.170260 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:27.170264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:27.170905 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:27.171107 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:27.171113 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.171838 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.171839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:27.172739 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:27.172741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.173140 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.173141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:27.173275 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:27.173771 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:27.173779 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:27.175197 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:27.176854 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:27.176861 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:27.176863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:27.177178 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:27.177398 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:27.177399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:27.177399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:27.177499 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:27.177871 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:27.177877 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:27.178007 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 19:25:27.178098 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 19:25:27.178232 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":139,"threads_started":1}
I20260809 19:25:27.178402 21929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:27.169174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:32787 (local address 127.21.81.254:41177)
0809 19:25:27.169222 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:27.169229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:27.169505 (+ 276us) server_negotiation.cc:407] Connection header received
0809 19:25:27.170112 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:27.170113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:27.170126 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:27.170192 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:27.171220 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.171734 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.172837 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.173049 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.173132 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:27.175288 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:27.175290 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:27.175290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:27.175297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:27.176791 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:27.177259 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:27.177260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:27.177261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:27.177328 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:27.177643 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:27.177644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:27.177645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:27.177796 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:27.177873 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:27.178156 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 19:25:27.178263 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:25:27.179059 22236 heartbeater.cc:347] Connected to a master server at 127.21.81.254:41177
I20260809 19:25:27.179150 22236 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:27.179376 22236 heartbeater.cc:511] Master 127.21.81.254:41177 requested a full tablet report, sending...
I20260809 19:25:27.179850 21886 ts_manager.cc:194] Registered new tserver with Master: ff182406e6774c9e92e2326fae8118c9 (127.21.81.194:39919)
I20260809 19:25:27.180238 21886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:32787
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:25:27.203809 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:41177
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:27.298498 22242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:27.298795 22242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:27.298835 22242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:27.299997 22242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:27.300033 22242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:27.300060 22242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:25:27.301462 22242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:41177
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:27.301710 22242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:27.301910 22242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:27.307425 22250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:27.307435 22247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:27.307600 22242 server_base.cc:1034] running on GCE node
W20260809 19:25:27.307469 22248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:27.307842 22242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:27.308084 22242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:27.309224 22242 hybrid_clock.cc:648] HybridClock initialized: now 1786303527309217 us; error 29 us; skew 500 ppm
Aug 09 19:25:27 dist-test-slave-njxd krb5kdc[21838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303527, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:27.311806 22242 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:25:27.312340 22242 webserver.cc:460] Webserver started at http://127.21.81.195:40929/ using document root <none> and password file <none>
I20260809 19:25:27.312580 22242 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:27.312647 22242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:27.312795 22242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:27.313928 22242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "ddf06df1dc1349c191e2dd36b45fe942"
format_stamp: "Formatted at 2026-08-09 19:25:27 on dist-test-slave-njxd"
server_key: "f69b4b0adcc8f403391e96c2cee36dcd"
server_key_iv: "0052907f05b890ca89248b7e7919db44"
server_key_version: "encryptionkey@0"
I20260809 19:25:27.314291 22242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "ddf06df1dc1349c191e2dd36b45fe942"
format_stamp: "Formatted at 2026-08-09 19:25:27 on dist-test-slave-njxd"
server_key: "f69b4b0adcc8f403391e96c2cee36dcd"
server_key_iv: "0052907f05b890ca89248b7e7919db44"
server_key_version: "encryptionkey@0"
I20260809 19:25:27.315681 22242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 19:25:27.316519 22257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:27.316711 22242 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:27.316833 22242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "ddf06df1dc1349c191e2dd36b45fe942"
format_stamp: "Formatted at 2026-08-09 19:25:27 on dist-test-slave-njxd"
server_key: "f69b4b0adcc8f403391e96c2cee36dcd"
server_key_iv: "0052907f05b890ca89248b7e7919db44"
server_key_version: "encryptionkey@0"
I20260809 19:25:27.316911 22242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:27.332945 22242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:27.335109 22242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:27.335284 22242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:27.335552 22242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:27.335897 22242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:27.335952 22242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:27.335999 22242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:27.336022 22242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:27.345897 22242 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:35949
I20260809 19:25:27.345961 22380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:35949 every 8 connection(s)
I20260809 19:25:27.346282 22242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:25:27.349400 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 22242
I20260809 19:25:27.349468 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:25:27.349658 21831 external_mini_cluster.cc:1428] Setting key dcb16120f6e2de291334bce8e4c947e7
Aug 09 19:25:27 dist-test-slave-njxd krb5kdc[21838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303527, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:27.356905 21929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:27.347075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:50461 (local address 127.21.81.254:41177)
0809 19:25:27.347159 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:27.347162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:27.347499 (+ 337us) server_negotiation.cc:407] Connection header received
0809 19:25:27.348250 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:27.348251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:27.348261 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:27.348302 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:27.349283 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.350004 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.351636 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.351829 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.351879 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:27.354223 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:27.354224 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:27.354224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:27.354231 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:27.355664 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:27.356130 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:27.356131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:27.356131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:27.356184 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:27.356404 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:27.356405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:27.356405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:27.356503 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:27.356552 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:27.356783 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 19:25:27.356842 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 19:25:27.357069 22385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:27.347060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:41177 (local address 127.21.81.195:50461)
0809 19:25:27.347359 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:27.347395 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:27.348136 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:27.348354 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:27.348356 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:27.348952 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:27.349193 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:27.349198 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.350123 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.350124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:27.351545 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:27.351549 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.351902 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.351903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:27.352052 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:27.352551 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:27.352560 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:27.354135 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:27.355837 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:27.355844 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:27.355846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:27.356061 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:27.356243 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:27.356244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:27.356244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:27.356348 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:27.356557 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:27.356562 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:27.356669 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 19:25:27.356753 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 19:25:27.356869 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":131,"threads_started":1}
I20260809 19:25:27.357643 22382 heartbeater.cc:347] Connected to a master server at 127.21.81.254:41177
I20260809 19:25:27.357769 22382 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:27.357946 22382 heartbeater.cc:511] Master 127.21.81.254:41177 requested a full tablet report, sending...
I20260809 19:25:27.358343 21886 ts_manager.cc:194] Registered new tserver with Master: ddf06df1dc1349c191e2dd36b45fe942 (127.21.81.195:35949)
I20260809 19:25:27.358668 21886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:50461
I20260809 19:25:27.361796 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 19:25:27 dist-test-slave-njxd krb5kdc[21838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303527, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 19:25:27 dist-test-slave-njxd krb5kdc[21838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303527, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.81.193@KRBTEST.COM
I20260809 19:25:27.386327 22391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:27.378708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35250 (local address 127.21.81.193:40455)
0809 19:25:27.378906 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:27.378929 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:27.378939 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:25:27.378980 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:27.378982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:27.378995 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:27.379173 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:27.379719 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.380436 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.381146 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.381456 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.381509 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:27.383791 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:27.383798 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:27.383798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:27.383806 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:27.385220 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:27.385524 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:27.385526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:27.385526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:27.385636 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:27.385808 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:27.385808 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:27.385809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:27.385967 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:27.386083 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:27.386130 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 19:25:27.386191 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":118,"threads_started":1}
W20260809 19:25:27.386735 21998 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35250
Aug 09 19:25:27 dist-test-slave-njxd krb5kdc[21838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 19:25:27 dist-test-slave-njxd krb5kdc[21838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.193@KRBTEST.COM
I20260809 19:25:27.410112 22391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:27.403132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35254 (local address 127.21.81.193:40455)
0809 19:25:27.403201 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:27.403203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:27.403308 (+ 105us) server_negotiation.cc:407] Connection header received
0809 19:25:27.403357 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:27.403358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:27.403367 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:27.403417 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:27.403966 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.404447 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.405098 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:27.405289 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:27.405338 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:27.407535 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:27.407536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:27.407537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:27.407546 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:27.408902 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:27.409278 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:27.409280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:27.409280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:27.409345 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:27.409541 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:27.409542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:27.409542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:27.409695 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:27.409832 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:27.409919 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 19:25:27.410002 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:25:27.410658 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 21952
I20260809 19:25:27.418550 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 22095
I20260809 19:25:27.426656 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 22242
I20260809 19:25:27.432271 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 21852
2026-08-09T19:25:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1976 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:27 dist-test-slave-njxd krb5kdc[22398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:27 dist-test-slave-njxd krb5kdc[22398](info): set up 2 sockets
Aug 09 19:25:27 dist-test-slave-njxd krb5kdc[22398](info): commencing operation
krb5kdc: starting...
W20260809 19:25:28.481026 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:25:28 dist-test-slave-njxd krb5kdc[22398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:25:28.600943 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39849
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:39849
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:28.682695 22413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:28.682934 22413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:28.682972 22413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:28.684132 22413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:28.684196 22413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:28.684219 22413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:28.684238 22413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:28.684265 22413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:28.685706 22413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:39849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:28.685977 22413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:28.686193 22413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:25:28.691043 22413 server_base.cc:1034] running on GCE node
W20260809 19:25:28.690971 22419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:28.690981 22418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:28.691169 22421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:28.691447 22413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:28.691730 22413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:28.692893 22413 hybrid_clock.cc:648] HybridClock initialized: now 1786303528692890 us; error 36 us; skew 500 ppm
Aug 09 19:25:28 dist-test-slave-njxd krb5kdc[22398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303528, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:28.695416 22413 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:25:28.695921 22413 webserver.cc:460] Webserver started at http://127.21.81.254:44967/ using document root <none> and password file <none>
I20260809 19:25:28.696117 22413 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:28.696157 22413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:28.696249 22413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:28.697229 22413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "1453154fef7d4f2fa5e50cf704a80ba9"
format_stamp: "Formatted at 2026-08-09 19:25:28 on dist-test-slave-njxd"
server_key: "82cc42f35b0d351aa2d1bc9e49d71cdb"
server_key_iv: "2500001b3dcf4d29c389c9fae867ab23"
server_key_version: "encryptionkey@0"
I20260809 19:25:28.697526 22413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "1453154fef7d4f2fa5e50cf704a80ba9"
format_stamp: "Formatted at 2026-08-09 19:25:28 on dist-test-slave-njxd"
server_key: "82cc42f35b0d351aa2d1bc9e49d71cdb"
server_key_iv: "2500001b3dcf4d29c389c9fae867ab23"
server_key_version: "encryptionkey@0"
I20260809 19:25:28.698866 22413 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:28.699517 22429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:28.699723 22413 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:28.699846 22413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "1453154fef7d4f2fa5e50cf704a80ba9"
format_stamp: "Formatted at 2026-08-09 19:25:28 on dist-test-slave-njxd"
server_key: "82cc42f35b0d351aa2d1bc9e49d71cdb"
server_key_iv: "2500001b3dcf4d29c389c9fae867ab23"
server_key_version: "encryptionkey@0"
I20260809 19:25:28.699915 22413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:28.717828 22413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:28.719661 22413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:28.719826 22413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:28.723811 22413 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:39849
I20260809 19:25:28.723891 22484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:39849 every 8 connection(s)
I20260809 19:25:28.724223 22413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:28.725128 22485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:28.726006 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 22413
I20260809 19:25:28.726079 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:28.726245 21831 external_mini_cluster.cc:1428] Setting key a8e668d971271f3088fb96b463fd36f1
I20260809 19:25:28.728325 22485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9: Bootstrap starting.
I20260809 19:25:28.729045 22485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:28.729363 22485 log.cc:824] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:28.730142 22485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9: No bootstrap required, opened a new log
I20260809 19:25:28.732517 22485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1453154fef7d4f2fa5e50cf704a80ba9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39849 } }
I20260809 19:25:28.732709 22485 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:28.732745 22485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1453154fef7d4f2fa5e50cf704a80ba9, State: Initialized, Role: FOLLOWER
I20260809 19:25:28.732879 22485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1453154fef7d4f2fa5e50cf704a80ba9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39849 } }
I20260809 19:25:28.732964 22485 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:28.732995 22485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:28.733040 22485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 19:25:28 dist-test-slave-njxd krb5kdc[22398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:28.733805 22485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1453154fef7d4f2fa5e50cf704a80ba9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39849 } }
I20260809 19:25:28.733896 22485 leader_election.cc:302] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1453154fef7d4f2fa5e50cf704a80ba9; no voters:
I20260809 19:25:28.734079 22485 leader_election.cc:288] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:28.734123 22491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:28.734340 22485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:28.734650 22491 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [term 1 LEADER]: Becoming Leader. State: Replica: 1453154fef7d4f2fa5e50cf704a80ba9, State: Running, Role: LEADER
I20260809 19:25:28.734736 22491 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1453154fef7d4f2fa5e50cf704a80ba9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39849 } }
I20260809 19:25:28.735087 22493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1453154fef7d4f2fa5e50cf704a80ba9. Latest consensus state: current_term: 1 leader_uuid: "1453154fef7d4f2fa5e50cf704a80ba9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1453154fef7d4f2fa5e50cf704a80ba9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39849 } } }
I20260809 19:25:28.735198 22493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:28.735404 22500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:28.735941 22500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:28.736153 22492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1453154fef7d4f2fa5e50cf704a80ba9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1453154fef7d4f2fa5e50cf704a80ba9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39849 } } }
I20260809 19:25:28.736344 22492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:28.737810 22500 catalog_manager.cc:1347] Generated new cluster ID: 38a3f8714b19433d80aa21a642b29f2c
I20260809 19:25:28.737864 22500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:28.738006 22488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:28.726800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.21.81.254:39849)
0809 19:25:28.727000 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:28.727009 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:28.727040 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:25:28.727535 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:28.727544 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:28.727606 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:28.727749 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:28.728316 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:28.729707 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:28.730519 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:28.731122 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:28.731250 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:28.733756 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:28.733759 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:28.733760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:28.733770 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:28.735872 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:28.736413 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:28.736416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:28.736417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:28.736520 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:28.736813 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:28.736814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:28.736815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:28.737111 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:28.737248 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:28.737644 (+ 396us) server_negotiation.cc:299] Negotiation successful
0809 19:25:28.737812 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260809 19:25:28.768731 22500 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:28.769295 22500 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:28.771128 22500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1453154fef7d4f2fa5e50cf704a80ba9: Generated new TSK 0
I20260809 19:25:28.771319 22500 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:25:28.804114 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39849
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:28.908994 22512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:28.909186 22512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:28.909210 22512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:28.910362 22512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:28.910404 22512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:28.910430 22512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:25:28.911921 22512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39849
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:28.912161 22512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:28.912405 22512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:28.917704 22520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:28.917706 22518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:28.917706 22517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:28.917874 22512 server_base.cc:1034] running on GCE node
I20260809 19:25:28.918221 22512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:28.918515 22512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:28.919690 22512 hybrid_clock.cc:648] HybridClock initialized: now 1786303528919685 us; error 41 us; skew 500 ppm
Aug 09 19:25:28 dist-test-slave-njxd krb5kdc[22398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303528, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:28.922230 22512 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:25:28.922664 22512 webserver.cc:460] Webserver started at http://127.21.81.193:42909/ using document root <none> and password file <none>
I20260809 19:25:28.922899 22512 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:28.922967 22512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:28.923120 22512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:28.924162 22512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "ca62d264e5734db2895c2d34306f2bbc"
format_stamp: "Formatted at 2026-08-09 19:25:28 on dist-test-slave-njxd"
server_key: "cfc02e594f6f43aa1b62d6d9fdba5a41"
server_key_iv: "9237bacf07f876869da1097c3e771c39"
server_key_version: "encryptionkey@0"
I20260809 19:25:28.924526 22512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "ca62d264e5734db2895c2d34306f2bbc"
format_stamp: "Formatted at 2026-08-09 19:25:28 on dist-test-slave-njxd"
server_key: "cfc02e594f6f43aa1b62d6d9fdba5a41"
server_key_iv: "9237bacf07f876869da1097c3e771c39"
server_key_version: "encryptionkey@0"
I20260809 19:25:28.925927 22512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:25:28.926937 22528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:28.927135 22512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:25:28.927228 22512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "ca62d264e5734db2895c2d34306f2bbc"
format_stamp: "Formatted at 2026-08-09 19:25:28 on dist-test-slave-njxd"
server_key: "cfc02e594f6f43aa1b62d6d9fdba5a41"
server_key_iv: "9237bacf07f876869da1097c3e771c39"
server_key_version: "encryptionkey@0"
I20260809 19:25:28.927284 22512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:28.933946 22512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:28.936074 22512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:28.936192 22512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:28.936368 22512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:28.936592 22512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:28.936620 22512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:28.936640 22512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:28.936653 22512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:28.942725 22512 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:37915
I20260809 19:25:28.942801 22646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:37915 every 8 connection(s)
I20260809 19:25:28.943121 22512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:25:28 dist-test-slave-njxd krb5kdc[22398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303528, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:28.949914 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 22512
I20260809 19:25:28.949997 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:25:28.950170 21831 external_mini_cluster.cc:1428] Setting key e5ea0473654569803148fcf3d790706b
I20260809 19:25:28.954782 22488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:28.943904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:38041 (local address 127.21.81.254:39849)
0809 19:25:28.943970 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:28.943972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:28.944405 (+ 433us) server_negotiation.cc:407] Connection header received
0809 19:25:28.945484 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:28.945485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:28.945499 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:28.945552 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:28.946224 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:28.946731 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:28.947737 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:28.947973 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:28.948076 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:28.950453 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:28.950454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:28.950454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:28.950462 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:28.952342 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:28.953201 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:28.953203 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:28.953203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:28.953298 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:28.953773 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:28.953775 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:28.953775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:28.953937 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:28.954015 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:28.954507 (+ 492us) server_negotiation.cc:299] Negotiation successful
0809 19:25:28.954643 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 19:25:28.955147 22650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:28.943870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39849 (local address 127.21.81.193:38041)
0809 19:25:28.944071 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:28.944283 (+ 212us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:28.945360 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:28.945664 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:28.945667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:28.945926 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:28.946112 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:28.946116 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:28.946829 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:28.946830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:28.947643 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:28.947645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:28.948086 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:28.948087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:28.948212 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:28.948714 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:28.948722 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:28.950337 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:28.952463 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:28.952471 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:28.952473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:28.952984 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:28.953429 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:28.953430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:28.953430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:28.953660 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:28.954020 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:28.954027 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:28.954287 (+ 260us) client_negotiation.cc:769] Sending connection context
0809 19:25:28.954450 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 19:25:28.954788 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":101,"threads_started":1}
I20260809 19:25:28.955838 22647 heartbeater.cc:347] Connected to a master server at 127.21.81.254:39849
I20260809 19:25:28.955952 22647 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:28.956146 22647 heartbeater.cc:511] Master 127.21.81.254:39849 requested a full tablet report, sending...
I20260809 19:25:28.956802 22446 ts_manager.cc:194] Registered new tserver with Master: ca62d264e5734db2895c2d34306f2bbc (127.21.81.193:37915)
I20260809 19:25:28.958252 22446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:38041
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:25:28.976823 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39849
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:29.058715 22653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:29.058954 22653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:29.058992 22653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:29.060146 22653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:29.060212 22653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:29.060256 22653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:25:29.061758 22653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39849
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:29.062021 22653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:29.062249 22653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:29.067055 22658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:29.067077 22659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:29.067061 22661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:29.067256 22653 server_base.cc:1034] running on GCE node
I20260809 19:25:29.067579 22653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:29.067834 22653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:29.068982 22653 hybrid_clock.cc:648] HybridClock initialized: now 1786303529068968 us; error 28 us; skew 500 ppm
Aug 09 19:25:29 dist-test-slave-njxd krb5kdc[22398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303529, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:29.071815 22653 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:25:29.072405 22653 webserver.cc:460] Webserver started at http://127.21.81.194:45709/ using document root <none> and password file <none>
I20260809 19:25:29.072654 22653 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:29.072700 22653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:29.072855 22653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:29.074016 22653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "f01348f5c4fb42e281cadc4ba36877e4"
format_stamp: "Formatted at 2026-08-09 19:25:29 on dist-test-slave-njxd"
server_key: "5fda5f25ad190013e27b05732038fcb3"
server_key_iv: "0af07cb10defbb9438b84961c97cd5e3"
server_key_version: "encryptionkey@0"
I20260809 19:25:29.074393 22653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "f01348f5c4fb42e281cadc4ba36877e4"
format_stamp: "Formatted at 2026-08-09 19:25:29 on dist-test-slave-njxd"
server_key: "5fda5f25ad190013e27b05732038fcb3"
server_key_iv: "0af07cb10defbb9438b84961c97cd5e3"
server_key_version: "encryptionkey@0"
I20260809 19:25:29.076601 22653 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 19:25:29.077540 22669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:29.077744 22653 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:29.077844 22653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "f01348f5c4fb42e281cadc4ba36877e4"
format_stamp: "Formatted at 2026-08-09 19:25:29 on dist-test-slave-njxd"
server_key: "5fda5f25ad190013e27b05732038fcb3"
server_key_iv: "0af07cb10defbb9438b84961c97cd5e3"
server_key_version: "encryptionkey@0"
I20260809 19:25:29.077917 22653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:29.089076 22653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:29.090801 22653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:29.090902 22653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:29.091063 22653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:29.091389 22653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:29.091444 22653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:29.091485 22653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:29.091514 22653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:29.097345 22653 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:36261
I20260809 19:25:29.097411 22792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:36261 every 8 connection(s)
I20260809 19:25:29.097746 22653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:25:29.101462 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 22653
I20260809 19:25:29.101552 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:25:29.101745 21831 external_mini_cluster.cc:1428] Setting key 75f0750f87332a39c8512f590a12d699
Aug 09 19:25:29 dist-test-slave-njxd krb5kdc[22398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303529, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:29.107815 22488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.098471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:58681 (local address 127.21.81.254:39849)
0809 19:25:29.098559 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:29.098561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:29.098731 (+ 170us) server_negotiation.cc:407] Connection header received
0809 19:25:29.099630 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:29.099631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:29.099641 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:29.099688 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:29.100272 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.100747 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.102060 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.102307 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.102374 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.104895 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:29.104897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:29.104897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:29.104904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:29.106359 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:29.106874 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:29.106875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:29.106875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:29.106926 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:29.107205 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:29.107205 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:29.107205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:29.107300 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:29.107349 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:29.107678 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 19:25:29.107753 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":55}
I20260809 19:25:29.108136 22795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.098426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39849 (local address 127.21.81.194:58681)
0809 19:25:29.098610 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:29.098644 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:29.099542 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:29.099751 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:29.099754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:29.100003 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:29.100185 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:29.100189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.100826 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.100826 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:29.101980 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:29.101983 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.102406 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.102408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:29.102564 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.103149 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:29.103157 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:29.104798 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:29.106445 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:29.106453 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:29.106456 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:29.106799 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:29.106993 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:29.106994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:29.106994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:29.107137 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:29.107362 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:29.107369 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:29.107539 (+ 170us) client_negotiation.cc:769] Sending connection context
0809 19:25:29.107663 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 19:25:29.107858 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":106,"threads_started":1}
I20260809 19:25:29.108697 22793 heartbeater.cc:347] Connected to a master server at 127.21.81.254:39849
I20260809 19:25:29.108809 22793 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:29.108982 22793 heartbeater.cc:511] Master 127.21.81.254:39849 requested a full tablet report, sending...
I20260809 19:25:29.109385 22446 ts_manager.cc:194] Registered new tserver with Master: f01348f5c4fb42e281cadc4ba36877e4 (127.21.81.194:36261)
I20260809 19:25:29.109804 22446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:58681
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:25:29.128463 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39849
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:29.210161 22798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:29.210389 22798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:29.210430 22798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:29.211611 22798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:29.211678 22798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:29.211722 22798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:25:29.213267 22798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39849
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:29.213529 22798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:29.213799 22798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:29.218824 22803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:29.218871 22804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:29.219023 22798 server_base.cc:1034] running on GCE node
W20260809 19:25:29.218874 22806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:29.219285 22798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:29.219529 22798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:29.220685 22798 hybrid_clock.cc:648] HybridClock initialized: now 1786303529220687 us; error 60 us; skew 500 ppm
Aug 09 19:25:29 dist-test-slave-njxd krb5kdc[22398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303529, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:29.223330 22798 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:25:29.223754 22798 webserver.cc:460] Webserver started at http://127.21.81.195:36023/ using document root <none> and password file <none>
I20260809 19:25:29.223971 22798 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:29.224015 22798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:29.224164 22798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:29.225260 22798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "e877291082384c039213e206b863aaa1"
format_stamp: "Formatted at 2026-08-09 19:25:29 on dist-test-slave-njxd"
server_key: "d063b19770335e0ba0ff801c73da98d3"
server_key_iv: "972162033566e40eb05ce0f2e62ac51c"
server_key_version: "encryptionkey@0"
I20260809 19:25:29.225692 22798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "e877291082384c039213e206b863aaa1"
format_stamp: "Formatted at 2026-08-09 19:25:29 on dist-test-slave-njxd"
server_key: "d063b19770335e0ba0ff801c73da98d3"
server_key_iv: "972162033566e40eb05ce0f2e62ac51c"
server_key_version: "encryptionkey@0"
I20260809 19:25:29.227119 22798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:25:29.227867 22813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:29.228052 22798 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:29.228142 22798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "e877291082384c039213e206b863aaa1"
format_stamp: "Formatted at 2026-08-09 19:25:29 on dist-test-slave-njxd"
server_key: "d063b19770335e0ba0ff801c73da98d3"
server_key_iv: "972162033566e40eb05ce0f2e62ac51c"
server_key_version: "encryptionkey@0"
I20260809 19:25:29.228194 22798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:29.254139 22798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:29.256659 22798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:29.256795 22798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:29.257040 22798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:29.257397 22798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:29.257436 22798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:29.257481 22798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:29.257511 22798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:29.267086 22798 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:42775
I20260809 19:25:29.267171 22934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:42775 every 8 connection(s)
I20260809 19:25:29.267482 22798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:25:29.273720 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 22798
I20260809 19:25:29.273837 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:25:29.273988 21831 external_mini_cluster.cc:1428] Setting key fa499bbd5a1974218ad5aa3659f0b2f9
Aug 09 19:25:29 dist-test-slave-njxd krb5kdc[22398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303529, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:29.278020 22488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.268215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:40709 (local address 127.21.81.254:39849)
0809 19:25:29.268282 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:29.268284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:29.268535 (+ 251us) server_negotiation.cc:407] Connection header received
0809 19:25:29.269621 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:29.269623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:29.269636 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:29.269701 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:29.270333 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.270810 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.272376 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.272740 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.272815 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.275575 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:29.275576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:29.275576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:29.275583 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:29.276788 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:29.277138 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:29.277139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:29.277139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:29.277189 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:29.277395 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:29.277396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:29.277396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:29.277511 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:29.277622 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:29.277894 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 19:25:29.277957 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:25:29.278262 22937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.268199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39849 (local address 127.21.81.195:40709)
0809 19:25:29.268417 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:29.268452 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:29.269493 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:29.269771 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:29.269775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:29.270044 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:29.270260 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:29.270264 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.270885 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.270886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:29.272281 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:29.272284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.272846 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.272847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:29.272982 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.273629 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:29.273639 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:29.275493 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:29.276857 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:29.276863 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:29.276865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:29.277074 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:29.277247 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:29.277247 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:29.277248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:29.277341 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:29.277634 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:29.277641 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:29.277778 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 19:25:29.277864 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 19:25:29.277981 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":139,"threads_started":1}
I20260809 19:25:29.278774 22935 heartbeater.cc:347] Connected to a master server at 127.21.81.254:39849
I20260809 19:25:29.278885 22935 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:29.279066 22935 heartbeater.cc:511] Master 127.21.81.254:39849 requested a full tablet report, sending...
I20260809 19:25:29.279398 22446 ts_manager.cc:194] Registered new tserver with Master: e877291082384c039213e206b863aaa1 (127.21.81.195:42775)
I20260809 19:25:29.279760 22446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:40709
I20260809 19:25:29.286125 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:25:29.293682 22488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.287908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36678 (local address 127.21.81.254:39849)
0809 19:25:29.288106 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:29.288108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:29.288118 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:25:29.288140 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:29.288142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:29.288152 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:29.288213 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:29.288780 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.289237 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.290028 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.290202 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.290257 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.291318 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:29.291319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:29.291319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:29.291325 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:29.292659 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:29.292953 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:29.292954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:29.292954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:29.293003 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:29.293174 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:29.293175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:29.293175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:29.293312 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:29.293387 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:29.293472 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 19:25:29.293529 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172}
I20260809 19:25:29.295773 22446 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:25:29.297271 22446 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:25:29.304176 22946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.299581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:37915 (local address 127.0.0.1:40212)
0809 19:25:29.299815 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:29.299826 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:29.299871 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:29.300459 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:29.300460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:29.300463 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:29.300568 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:29.300570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.301664 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.301665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:29.302868 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:29.302871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.303818 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.303819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:29.303900 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.303916 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 19:25:29.304017 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 19:25:29.304081 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":43,"thread_start_us":74,"threads_started":1}
I20260809 19:25:29.304184 22950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.300084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.21.81.193:37915)
0809 19:25:29.300285 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:29.300289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:29.300300 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:25:29.300339 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:29.300342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:29.300356 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:29.300448 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:29.300722 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.301468 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.302970 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.303701 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.303806 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.303873 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:29.304047 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 19:25:29.304099 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":94,"threads_started":1}
I20260809 19:25:29.304267 22948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.299663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:42775 (local address 127.0.0.1:53710)
0809 19:25:29.299958 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:29.299966 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:29.300000 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:29.300730 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:29.300731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:29.300733 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:29.300791 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:29.300793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.302158 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.302159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:29.303278 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:29.303280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.304077 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.304078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:29.304134 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.304138 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:29.304166 (+ 28us) client_negotiation.cc:240] Negotiation successful
0809 19:25:29.304203 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":110,"threads_started":1}
I20260809 19:25:29.304593 22949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.299829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35722 (local address 127.21.81.194:36261)
0809 19:25:29.299987 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:29.299991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:29.300003 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:25:29.300071 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:29.300073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:29.300089 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:29.300187 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:29.300354 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.301512 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.303126 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.303790 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.303849 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.303895 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:29.304236 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 19:25:29.304513 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260809 19:25:29.304194 22947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.299581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:36261 (local address 127.0.0.1:35722)
0809 19:25:29.299766 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:29.299804 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:29.299871 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:29.300216 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:29.300219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:29.300221 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:29.300288 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:29.300289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.301662 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.301664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:29.303042 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:29.303045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.303939 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.303940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:29.304011 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.304015 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:29.304080 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 19:25:29.304120 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":54,"thread_start_us":68,"threads_started":1}
I20260809 19:25:29.304708 22951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.300319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53710 (local address 127.21.81.195:42775)
0809 19:25:29.300539 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:29.300544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:29.300555 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:25:29.300601 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:29.300602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:29.300617 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:29.300859 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:29.300928 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.301996 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.303351 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.303981 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.304423 (+ 442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.304514 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:29.304542 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 19:25:29.304592 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":92,"threads_started":1}
I20260809 19:25:29.305168 22578 tablet_service.cc:1467] Processing CreateTablet for tablet 3fb49465051d47ce98ce2331eb708c31 (DEFAULT_TABLE table=test-table [id=bc35a7ee4e9241e7ad5372462eb4ee85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:29.305287 22865 tablet_service.cc:1467] Processing CreateTablet for tablet 3fb49465051d47ce98ce2331eb708c31 (DEFAULT_TABLE table=test-table [id=bc35a7ee4e9241e7ad5372462eb4ee85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:29.305449 22721 tablet_service.cc:1467] Processing CreateTablet for tablet 3fb49465051d47ce98ce2331eb708c31 (DEFAULT_TABLE table=test-table [id=bc35a7ee4e9241e7ad5372462eb4ee85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:29.305605 22578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fb49465051d47ce98ce2331eb708c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:29.305584 22865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fb49465051d47ce98ce2331eb708c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:29.305840 22721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fb49465051d47ce98ce2331eb708c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:29.309022 22952 tablet_bootstrap.cc:492] T 3fb49465051d47ce98ce2331eb708c31 P f01348f5c4fb42e281cadc4ba36877e4: Bootstrap starting.
I20260809 19:25:29.309446 22953 tablet_bootstrap.cc:492] T 3fb49465051d47ce98ce2331eb708c31 P e877291082384c039213e206b863aaa1: Bootstrap starting.
I20260809 19:25:29.309687 22954 tablet_bootstrap.cc:492] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc: Bootstrap starting.
I20260809 19:25:29.310196 22952 tablet_bootstrap.cc:654] T 3fb49465051d47ce98ce2331eb708c31 P f01348f5c4fb42e281cadc4ba36877e4: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:29.310478 22953 tablet_bootstrap.cc:654] T 3fb49465051d47ce98ce2331eb708c31 P e877291082384c039213e206b863aaa1: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:29.310567 22954 tablet_bootstrap.cc:654] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:29.311123 22954 log.cc:824] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:29.311235 22952 log.cc:824] T 3fb49465051d47ce98ce2331eb708c31 P f01348f5c4fb42e281cadc4ba36877e4: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:29.311367 22953 log.cc:824] T 3fb49465051d47ce98ce2331eb708c31 P e877291082384c039213e206b863aaa1: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:29.312083 22954 tablet_bootstrap.cc:492] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc: No bootstrap required, opened a new log
I20260809 19:25:29.312139 22952 tablet_bootstrap.cc:492] T 3fb49465051d47ce98ce2331eb708c31 P f01348f5c4fb42e281cadc4ba36877e4: No bootstrap required, opened a new log
I20260809 19:25:29.312157 22954 ts_tablet_manager.cc:1397] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:25:29.312214 22952 ts_tablet_manager.cc:1397] T 3fb49465051d47ce98ce2331eb708c31 P f01348f5c4fb42e281cadc4ba36877e4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:25:29.314067 22953 tablet_bootstrap.cc:492] T 3fb49465051d47ce98ce2331eb708c31 P e877291082384c039213e206b863aaa1: No bootstrap required, opened a new log
I20260809 19:25:29.314215 22953 ts_tablet_manager.cc:1397] T 3fb49465051d47ce98ce2331eb708c31 P e877291082384c039213e206b863aaa1: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 19:25:29.314424 22952 raft_consensus.cc:348] T 3fb49465051d47ce98ce2331eb708c31 P f01348f5c4fb42e281cadc4ba36877e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca62d264e5734db2895c2d34306f2bbc" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37915 } } peers { permanent_uuid: "f01348f5c4fb42e281cadc4ba36877e4" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 36261 } } peers { permanent_uuid: "e877291082384c039213e206b863aaa1" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42775 } }
I20260809 19:25:29.314440 22954 raft_consensus.cc:348] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca62d264e5734db2895c2d34306f2bbc" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37915 } } peers { permanent_uuid: "f01348f5c4fb42e281cadc4ba36877e4" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 36261 } } peers { permanent_uuid: "e877291082384c039213e206b863aaa1" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42775 } }
I20260809 19:25:29.314628 22952 raft_consensus.cc:374] T 3fb49465051d47ce98ce2331eb708c31 P f01348f5c4fb42e281cadc4ba36877e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:29.314670 22952 raft_consensus.cc:729] T 3fb49465051d47ce98ce2331eb708c31 P f01348f5c4fb42e281cadc4ba36877e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f01348f5c4fb42e281cadc4ba36877e4, State: Initialized, Role: FOLLOWER
I20260809 19:25:29.314661 22954 raft_consensus.cc:374] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:29.314711 22954 raft_consensus.cc:729] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca62d264e5734db2895c2d34306f2bbc, State: Initialized, Role: FOLLOWER
I20260809 19:25:29.314836 22952 consensus_queue.cc:260] T 3fb49465051d47ce98ce2331eb708c31 P f01348f5c4fb42e281cadc4ba36877e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca62d264e5734db2895c2d34306f2bbc" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37915 } } peers { permanent_uuid: "f01348f5c4fb42e281cadc4ba36877e4" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 36261 } } peers { permanent_uuid: "e877291082384c039213e206b863aaa1" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42775 } }
I20260809 19:25:29.314884 22954 consensus_queue.cc:260] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca62d264e5734db2895c2d34306f2bbc" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37915 } } peers { permanent_uuid: "f01348f5c4fb42e281cadc4ba36877e4" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 36261 } } peers { permanent_uuid: "e877291082384c039213e206b863aaa1" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42775 } }
I20260809 19:25:29.315209 22952 ts_tablet_manager.cc:1428] T 3fb49465051d47ce98ce2331eb708c31 P f01348f5c4fb42e281cadc4ba36877e4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:25:29.315227 22954 ts_tablet_manager.cc:1428] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:25:29.315249 22647 heartbeater.cc:503] Master 127.21.81.254:39849 was elected leader, sending a full tablet report...
I20260809 19:25:29.315341 22793 heartbeater.cc:503] Master 127.21.81.254:39849 was elected leader, sending a full tablet report...
I20260809 19:25:29.316901 22953 raft_consensus.cc:348] T 3fb49465051d47ce98ce2331eb708c31 P e877291082384c039213e206b863aaa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca62d264e5734db2895c2d34306f2bbc" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37915 } } peers { permanent_uuid: "f01348f5c4fb42e281cadc4ba36877e4" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 36261 } } peers { permanent_uuid: "e877291082384c039213e206b863aaa1" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42775 } }
I20260809 19:25:29.317065 22953 raft_consensus.cc:374] T 3fb49465051d47ce98ce2331eb708c31 P e877291082384c039213e206b863aaa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:29.317088 22953 raft_consensus.cc:729] T 3fb49465051d47ce98ce2331eb708c31 P e877291082384c039213e206b863aaa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e877291082384c039213e206b863aaa1, State: Initialized, Role: FOLLOWER
I20260809 19:25:29.317193 22953 consensus_queue.cc:260] T 3fb49465051d47ce98ce2331eb708c31 P e877291082384c039213e206b863aaa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca62d264e5734db2895c2d34306f2bbc" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37915 } } peers { permanent_uuid: "f01348f5c4fb42e281cadc4ba36877e4" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 36261 } } peers { permanent_uuid: "e877291082384c039213e206b863aaa1" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42775 } }
I20260809 19:25:29.317471 22953 ts_tablet_manager.cc:1428] T 3fb49465051d47ce98ce2331eb708c31 P e877291082384c039213e206b863aaa1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:25:29.317512 22935 heartbeater.cc:503] Master 127.21.81.254:39849 was elected leader, sending a full tablet report...
W20260809 19:25:29.348445 22794 tablet.cc:2369] T 3fb49465051d47ce98ce2331eb708c31 P f01348f5c4fb42e281cadc4ba36877e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:25:29.371491 22959 raft_consensus.cc:482] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:25:29.371627 22959 raft_consensus.cc:504] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca62d264e5734db2895c2d34306f2bbc" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37915 } } peers { permanent_uuid: "f01348f5c4fb42e281cadc4ba36877e4" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 36261 } } peers { permanent_uuid: "e877291082384c039213e206b863aaa1" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42775 } }
I20260809 19:25:29.372151 22959 leader_election.cc:288] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f01348f5c4fb42e281cadc4ba36877e4 (127.21.81.194:36261), e877291082384c039213e206b863aaa1 (127.21.81.195:42775)
I20260809 19:25:29.375403 22650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.372365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:42775 (local address 127.21.81.193:51233)
0809 19:25:29.372444 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:29.372457 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:29.372504 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:29.372701 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:29.372702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:29.372704 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:29.372783 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:29.372785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.373727 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.373728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:29.374499 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:29.374501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.375090 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.375091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:29.375208 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.375211 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:25:29.375286 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 19:25:29.375329 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 19:25:29.375403 22951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.372503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:51233 (local address 127.21.81.195:42775)
0809 19:25:29.372557 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:29.372560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:29.372571 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:25:29.372602 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:29.372603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:29.372613 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:29.372679 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:29.372891 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.373639 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.374609 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.374999 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.375091 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.375100 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:29.375293 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 19:25:29.375333 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:25:29.375692 22949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.372483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:49243 (local address 127.21.81.194:36261)
0809 19:25:29.372531 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:29.372533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:29.372958 (+ 425us) server_negotiation.cc:407] Connection header received
0809 19:25:29.373098 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:29.373100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:29.373118 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:29.373163 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:29.373374 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.373836 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.374947 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.375342 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.375426 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.375433 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:29.375589 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 19:25:29.375628 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:25:29.375710 22961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.372367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:36261 (local address 127.21.81.193:49243)
0809 19:25:29.372808 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:29.372817 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:29.372990 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:29.373193 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:29.373194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:29.373196 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:29.373306 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:29.373308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.373924 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.373925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:29.374847 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:29.374849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.375454 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.375455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:29.375525 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.375530 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:25:29.375582 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 19:25:29.375620 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":189,"thread_start_us":158,"threads_started":1}
I20260809 19:25:29.375797 22886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb49465051d47ce98ce2331eb708c31" candidate_uuid: "ca62d264e5734db2895c2d34306f2bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e877291082384c039213e206b863aaa1" is_pre_election: true
I20260809 19:25:29.375965 22886 raft_consensus.cc:2454] T 3fb49465051d47ce98ce2331eb708c31 P e877291082384c039213e206b863aaa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca62d264e5734db2895c2d34306f2bbc in term 0.
I20260809 19:25:29.376047 22743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb49465051d47ce98ce2331eb708c31" candidate_uuid: "ca62d264e5734db2895c2d34306f2bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01348f5c4fb42e281cadc4ba36877e4" is_pre_election: true
I20260809 19:25:29.376189 22743 raft_consensus.cc:2454] T 3fb49465051d47ce98ce2331eb708c31 P f01348f5c4fb42e281cadc4ba36877e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca62d264e5734db2895c2d34306f2bbc in term 0.
I20260809 19:25:29.376199 22531 leader_election.cc:302] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca62d264e5734db2895c2d34306f2bbc, e877291082384c039213e206b863aaa1; no voters:
I20260809 19:25:29.376410 22959 raft_consensus.cc:2781] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:25:29.376471 22959 raft_consensus.cc:482] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:25:29.376502 22959 raft_consensus.cc:3037] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:29.377290 22959 raft_consensus.cc:504] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca62d264e5734db2895c2d34306f2bbc" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37915 } } peers { permanent_uuid: "f01348f5c4fb42e281cadc4ba36877e4" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 36261 } } peers { permanent_uuid: "e877291082384c039213e206b863aaa1" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42775 } }
I20260809 19:25:29.377437 22959 leader_election.cc:288] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [CANDIDATE]: Term 1 election: Requested vote from peers f01348f5c4fb42e281cadc4ba36877e4 (127.21.81.194:36261), e877291082384c039213e206b863aaa1 (127.21.81.195:42775)
I20260809 19:25:29.377624 22886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb49465051d47ce98ce2331eb708c31" candidate_uuid: "ca62d264e5734db2895c2d34306f2bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e877291082384c039213e206b863aaa1"
I20260809 19:25:29.377621 22743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb49465051d47ce98ce2331eb708c31" candidate_uuid: "ca62d264e5734db2895c2d34306f2bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01348f5c4fb42e281cadc4ba36877e4"
I20260809 19:25:29.377713 22743 raft_consensus.cc:3037] T 3fb49465051d47ce98ce2331eb708c31 P f01348f5c4fb42e281cadc4ba36877e4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:29.377713 22886 raft_consensus.cc:3037] T 3fb49465051d47ce98ce2331eb708c31 P e877291082384c039213e206b863aaa1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:29.378871 22743 raft_consensus.cc:2454] T 3fb49465051d47ce98ce2331eb708c31 P f01348f5c4fb42e281cadc4ba36877e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca62d264e5734db2895c2d34306f2bbc in term 1.
I20260809 19:25:29.379004 22886 raft_consensus.cc:2454] T 3fb49465051d47ce98ce2331eb708c31 P e877291082384c039213e206b863aaa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca62d264e5734db2895c2d34306f2bbc in term 1.
I20260809 19:25:29.379089 22529 leader_election.cc:302] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca62d264e5734db2895c2d34306f2bbc, f01348f5c4fb42e281cadc4ba36877e4; no voters:
I20260809 19:25:29.379186 22959 raft_consensus.cc:2781] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:29.379498 22959 raft_consensus.cc:686] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [term 1 LEADER]: Becoming Leader. State: Replica: ca62d264e5734db2895c2d34306f2bbc, State: Running, Role: LEADER
I20260809 19:25:29.379623 22959 consensus_queue.cc:237] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca62d264e5734db2895c2d34306f2bbc" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37915 } } peers { permanent_uuid: "f01348f5c4fb42e281cadc4ba36877e4" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 36261 } } peers { permanent_uuid: "e877291082384c039213e206b863aaa1" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42775 } }
I20260809 19:25:29.380483 22446 catalog_manager.cc:5495] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc reported cstate change: term changed from 0 to 1, leader changed from <none> to ca62d264e5734db2895c2d34306f2bbc (127.21.81.193). New cstate: current_term: 1 leader_uuid: "ca62d264e5734db2895c2d34306f2bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca62d264e5734db2895c2d34306f2bbc" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f01348f5c4fb42e281cadc4ba36877e4" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e877291082384c039213e206b863aaa1" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42775 } health_report { overall_health: UNKNOWN } } }
Aug 09 19:25:29 dist-test-slave-njxd krb5kdc[22398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.193@KRBTEST.COM
I20260809 19:25:29.400081 22950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.392829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40226 (local address 127.21.81.193:37915)
0809 19:25:29.392888 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:29.392890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:29.392949 (+ 59us) server_negotiation.cc:407] Connection header received
0809 19:25:29.392999 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:29.392999 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:29.393008 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:29.393052 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:29.393626 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.394068 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.394766 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.394942 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.394991 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.397364 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:29.397373 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:29.397373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:29.397379 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:29.399016 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:29.399331 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:29.399334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:29.399334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:29.399442 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:29.399608 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:29.399608 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:29.399609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:29.399749 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:29.399860 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:29.399921 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 19:25:29.399982 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 19:25:29 dist-test-slave-njxd krb5kdc[22398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303529, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 19:25:29 dist-test-slave-njxd krb5kdc[22398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303529, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.81.193@KRBTEST.COM
I20260809 19:25:29.425115 22950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.417523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40240 (local address 127.21.81.193:37915)
0809 19:25:29.417613 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:29.417616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:29.417748 (+ 132us) server_negotiation.cc:407] Connection header received
0809 19:25:29.417822 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:29.417823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:29.417833 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:29.417904 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:29.418465 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.418928 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.419665 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.419868 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.419945 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.422425 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:29.422426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:29.422427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:29.422433 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:29.423857 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:29.424261 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:29.424262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:29.424262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:29.424324 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:29.424523 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:29.424525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:29.424525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:29.424628 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:29.424744 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:29.424925 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 19:25:29.425008 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 19:25:29 dist-test-slave-njxd krb5kdc[22398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260809 19:25:29.443857 22648 tablet.cc:2369] T 3fb49465051d47ce98ce2331eb708c31 P ca62d264e5734db2895c2d34306f2bbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 19:25:29 dist-test-slave-njxd krb5kdc[22398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.193@KRBTEST.COM
I20260809 19:25:29.449744 22950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:29.442160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40254 (local address 127.21.81.193:37915)
0809 19:25:29.442264 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:29.442267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:29.442350 (+ 83us) server_negotiation.cc:407] Connection header received
0809 19:25:29.442438 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:29.442439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:29.442456 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:29.442522 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:29.443097 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.443576 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.444410 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:29.444600 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:29.444674 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:29.447051 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:29.447052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:29.447053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:29.447061 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:29.448550 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:29.448942 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:29.448944 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:29.448944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:29.449005 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:29.449204 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:29.449205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:29.449205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:29.449318 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:29.449440 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:29.449534 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 19:25:29.449638 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 19:25:29.450246 22558 tablet_service.cc:2908] Rejecting scan request for tablet 3fb49465051d47ce98ce2331eb708c31: Uninitialized: clean time has not yet been initialized
I20260809 19:25:29.451083 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 22512
I20260809 19:25:29.458768 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 22653
I20260809 19:25:29.464287 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 22798
I20260809 19:25:29.470224 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 22413
2026-08-09T19:25:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2037 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:29 dist-test-slave-njxd krb5kdc[22980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:29 dist-test-slave-njxd krb5kdc[22980](info): set up 2 sockets
Aug 09 19:25:29 dist-test-slave-njxd krb5kdc[22980](info): commencing operation
krb5kdc: starting...
W20260809 19:25:30.533741 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:25:30 dist-test-slave-njxd krb5kdc[22980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:25:30.656388 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:40401
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:40401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:30.737265 22995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:30.737506 22995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:30.737545 22995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:30.738708 22995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:30.738780 22995 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 19:25:30.738802 22995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:30.738824 22995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:30.738853 22995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:30.738879 22995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:30.740381 22995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:40401
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:40401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:30.740648 22995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:30.740885 22995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:30.745596 23001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:30.745608 23002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:30.745608 23004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:30.745868 22995 server_base.cc:1034] running on GCE node
I20260809 19:25:30.746165 22995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:30.746500 22995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:30.747646 22995 hybrid_clock.cc:648] HybridClock initialized: now 1786303530747628 us; error 39 us; skew 500 ppm
Aug 09 19:25:30 dist-test-slave-njxd krb5kdc[22980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303530, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:30.757072 22995 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:25:30.757516 22995 webserver.cc:460] Webserver started at http://127.21.81.254:40393/ using document root <none> and password file <none>
I20260809 19:25:30.757789 22995 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:30.757862 22995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:30.758014 22995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:30.759063 22995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "8f2982bc21a54566952402ab85c2e08b"
format_stamp: "Formatted at 2026-08-09 19:25:30 on dist-test-slave-njxd"
server_key: "3139dda00f2fd98824e38e9b5c19d3a4"
server_key_iv: "c648eecdb6cbad9f0e7d89441c18bacf"
server_key_version: "encryptionkey@0"
I20260809 19:25:30.759433 22995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "8f2982bc21a54566952402ab85c2e08b"
format_stamp: "Formatted at 2026-08-09 19:25:30 on dist-test-slave-njxd"
server_key: "3139dda00f2fd98824e38e9b5c19d3a4"
server_key_iv: "c648eecdb6cbad9f0e7d89441c18bacf"
server_key_version: "encryptionkey@0"
I20260809 19:25:30.760748 22995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:25:30.761552 23011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:30.761785 22995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:30.761888 22995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "8f2982bc21a54566952402ab85c2e08b"
format_stamp: "Formatted at 2026-08-09 19:25:30 on dist-test-slave-njxd"
server_key: "3139dda00f2fd98824e38e9b5c19d3a4"
server_key_iv: "c648eecdb6cbad9f0e7d89441c18bacf"
server_key_version: "encryptionkey@0"
I20260809 19:25:30.761976 22995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:30.783741 22995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:30.786026 22995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:30.786207 22995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:30.791332 22995 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:40401
I20260809 19:25:30.791450 23076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:40401 every 8 connection(s)
I20260809 19:25:30.791770 22995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:30.791931 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 22995
I20260809 19:25:30.792042 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:30.792250 21831 external_mini_cluster.cc:1428] Setting key 1b13f78a2505f3a20ec9a4b17633f98e
I20260809 19:25:30.792610 23078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:30.796005 23078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b: Bootstrap starting.
I20260809 19:25:30.796674 23078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:30.796972 23078 log.cc:824] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:30.797842 23078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b: No bootstrap required, opened a new log
Aug 09 19:25:30 dist-test-slave-njxd krb5kdc[22980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:30.800397 23078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f2982bc21a54566952402ab85c2e08b" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40401 } }
I20260809 19:25:30.800575 23078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:30.800628 23078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f2982bc21a54566952402ab85c2e08b, State: Initialized, Role: FOLLOWER
I20260809 19:25:30.800791 23078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f2982bc21a54566952402ab85c2e08b" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40401 } }
I20260809 19:25:30.800871 23078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:30.800918 23078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:30.800966 23078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:30.801767 23078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f2982bc21a54566952402ab85c2e08b" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40401 } }
I20260809 19:25:30.801883 23078 leader_election.cc:302] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f2982bc21a54566952402ab85c2e08b; no voters:
I20260809 19:25:30.802052 23078 leader_election.cc:288] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:30.802083 23084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:30.802260 23084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [term 1 LEADER]: Becoming Leader. State: Replica: 8f2982bc21a54566952402ab85c2e08b, State: Running, Role: LEADER
I20260809 19:25:30.802390 23078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:30.802398 23084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f2982bc21a54566952402ab85c2e08b" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40401 } }
I20260809 19:25:30.802852 23086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f2982bc21a54566952402ab85c2e08b. Latest consensus state: current_term: 1 leader_uuid: "8f2982bc21a54566952402ab85c2e08b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f2982bc21a54566952402ab85c2e08b" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40401 } } }
I20260809 19:25:30.803095 23086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:30.803664 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:30.792798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36104 (local address 127.21.81.254:40401)
0809 19:25:30.792966 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:30.792981 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:30.793007 (+ 26us) server_negotiation.cc:407] Connection header received
0809 19:25:30.793427 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:30.793433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:30.793479 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:30.793638 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:30.794380 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:30.795287 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:30.796313 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:30.796690 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:30.796822 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:30.799383 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:30.799387 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:30.799387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:30.799399 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:30.801370 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:30.801775 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:30.801778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:30.801779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:30.801885 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:30.802113 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:30.802114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:30.802115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:30.802496 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:30.802999 (+ 503us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:30.803090 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 19:25:30.803272 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260809 19:25:30.803632 23085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f2982bc21a54566952402ab85c2e08b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f2982bc21a54566952402ab85c2e08b" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40401 } } }
I20260809 19:25:30.803995 23085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:30.803687 23093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:30.804608 23093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:30.806046 23093 catalog_manager.cc:1347] Generated new cluster ID: c56a5e494efd4a3ebe3ebf2823f9ccf9
I20260809 19:25:30.806104 23093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:30.813388 23093 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:30.814112 23093 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:30.822479 23093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b: Generated new TSK 0
I20260809 19:25:30.822664 23093 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:25:30.858506 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:40401
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:30.942013 23105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:30.942250 23105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:30.942293 23105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:30.943480 23105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:30.943567 23105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:30.943604 23105 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:25:30.943640 23105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:25:30.945102 23105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.81.254:40401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:30.945380 23105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:30.945626 23105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:30.950476 23113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:30.950482 23110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:30.950531 23111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:30.950874 23105 server_base.cc:1034] running on GCE node
I20260809 19:25:30.951155 23105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:30.951442 23105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:30.952601 23105 hybrid_clock.cc:648] HybridClock initialized: now 1786303530952578 us; error 27 us; skew 500 ppm
Aug 09 19:25:30 dist-test-slave-njxd krb5kdc[22980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303530, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:30.955112 23105 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:25:30.955530 23105 webserver.cc:460] Webserver started at http://127.21.81.193:46471/ using document root <none> and password file <none>
I20260809 19:25:30.955778 23105 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:30.955845 23105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:30.956000 23105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:30.957015 23105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "d439dd42f8d44cc8936df6fa9ecf5c72"
format_stamp: "Formatted at 2026-08-09 19:25:30 on dist-test-slave-njxd"
server_key: "fafbc82905c582b029c887f3b23cd7e7"
server_key_iv: "bc09d9a5bb47d4b1cf121365ae2f2c1b"
server_key_version: "encryptionkey@0"
I20260809 19:25:30.957384 23105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "d439dd42f8d44cc8936df6fa9ecf5c72"
format_stamp: "Formatted at 2026-08-09 19:25:30 on dist-test-slave-njxd"
server_key: "fafbc82905c582b029c887f3b23cd7e7"
server_key_iv: "bc09d9a5bb47d4b1cf121365ae2f2c1b"
server_key_version: "encryptionkey@0"
I20260809 19:25:30.958757 23105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:30.959474 23121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:30.959640 23105 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:30.959748 23105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "d439dd42f8d44cc8936df6fa9ecf5c72"
format_stamp: "Formatted at 2026-08-09 19:25:30 on dist-test-slave-njxd"
server_key: "fafbc82905c582b029c887f3b23cd7e7"
server_key_iv: "bc09d9a5bb47d4b1cf121365ae2f2c1b"
server_key_version: "encryptionkey@0"
I20260809 19:25:30.959828 23105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:30.974727 23105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:30.976440 23105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:30.976600 23105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:30.977077 23105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:30.977141 23105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:30.977181 23105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:30.977224 23105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:25:30 dist-test-slave-njxd krb5kdc[22980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303530, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:30.984637 23105 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:44423
I20260809 19:25:30.984725 23245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:44423 every 8 connection(s)
I20260809 19:25:30.985061 23105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:25:30.986835 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:30.977829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:47839 (local address 127.21.81.254:40401)
0809 19:25:30.977876 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:30.977878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:30.978144 (+ 266us) server_negotiation.cc:407] Connection header received
0809 19:25:30.978676 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:30.978677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:30.978690 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:30.978760 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:30.979609 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:30.980433 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:30.981554 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:30.981812 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:30.981891 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:30.984077 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:30.984078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:30.984079 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:30.984087 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:30.985728 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:30.986068 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:30.986068 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:30.986069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:30.986156 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:30.986349 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:30.986350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:30.986350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:30.986449 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:30.986497 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:30.986703 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 19:25:30.986768 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:25:30.986995 23132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:30.977848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:40401 (local address 127.21.81.193:47839)
0809 19:25:30.978023 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:30.978057 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:30.978579 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:30.978831 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:30.978835 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:30.979211 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:30.979487 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:30.979495 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:30.980545 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:30.980546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:30.981443 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:30.981445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:30.981924 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:30.981925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:30.982048 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:30.982559 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:30.982569 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:30.983962 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:30.985793 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:30.985797 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:30.985799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:30.986003 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:30.986199 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:30.986199 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:30.986199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:30.986298 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:30.986488 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:30.986491 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:30.986594 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 19:25:30.986667 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 19:25:30.986782 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
I20260809 19:25:30.987514 23246 heartbeater.cc:347] Connected to a master server at 127.21.81.254:40401
I20260809 19:25:30.987594 23246 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:30.987793 23246 heartbeater.cc:511] Master 127.21.81.254:40401 requested a full tablet report, sending...
I20260809 19:25:30.988296 23027 ts_manager.cc:194] Registered new tserver with Master: d439dd42f8d44cc8936df6fa9ecf5c72 (127.21.81.193:44423)
I20260809 19:25:30.989203 23027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:47839
I20260809 19:25:30.994140 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:30.988599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:37275 (local address 127.21.81.254:40401)
0809 19:25:30.988652 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:30.988654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:30.988711 (+ 57us) server_negotiation.cc:407] Connection header received
0809 19:25:30.988772 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:30.988773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:30.988780 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:30.988819 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:30.989151 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:30.989711 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:30.990391 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:30.990592 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:30.990665 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:30.991584 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:30.991586 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:30.991586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:30.991593 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:30.992996 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:30.993343 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:30.993344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:30.993344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:30.993395 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:30.993599 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:30.993600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:30.993600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:30.993725 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:30.993811 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:30.993976 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 19:25:30.994049 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:25:30.994150 23252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:30.988506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:40401 (local address 127.21.81.193:37275)
0809 19:25:30.988650 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:30.988661 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:30.988720 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:30.988831 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:30.988831 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:30.989016 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:30.989086 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:30.989087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:30.989809 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:30.989810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:30.990310 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:30.990311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:30.990664 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:30.990665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:30.990747 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:30.991166 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:30.991170 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:30.991479 (+ 309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:30.993097 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:30.993098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:30.993098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:30.993274 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:30.993466 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:30.993466 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:30.993467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:30.993519 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:30.993804 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:30.993805 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:30.993867 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 19:25:30.993951 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 19:25:30.994050 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260809 19:25:30.994390 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 23105
I20260809 19:25:30.994493 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:25:30.994702 21831 external_mini_cluster.cc:1428] Setting key d0d1e2032fefa89a03e2add99816fdcd
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:25:31.019974 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:40401
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:31.102725 23255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:31.102913 23255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:31.102936 23255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:31.104066 23255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:31.104105 23255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:31.104125 23255 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:25:31.104141 23255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:25:31.105453 23255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.81.254:40401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:31.105680 23255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:31.105860 23255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:31.110569 23263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:31.110528 23260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:31.110545 23261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:31.110765 23255 server_base.cc:1034] running on GCE node
I20260809 19:25:31.110955 23255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:31.111224 23255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:31.112381 23255 hybrid_clock.cc:648] HybridClock initialized: now 1786303531112374 us; error 34 us; skew 500 ppm
Aug 09 19:25:31 dist-test-slave-njxd krb5kdc[22980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303531, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:31.114794 23255 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:25:31.115260 23255 webserver.cc:460] Webserver started at http://127.21.81.194:32995/ using document root <none> and password file <none>
I20260809 19:25:31.115478 23255 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:31.115545 23255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:31.115705 23255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:31.116688 23255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "69914967a446474b8cffd38632e2f5c6"
format_stamp: "Formatted at 2026-08-09 19:25:31 on dist-test-slave-njxd"
server_key: "2748abaf89a5e1c9cce158ca797c733e"
server_key_iv: "d4fe9fb3dd682a44978eaa7b01023774"
server_key_version: "encryptionkey@0"
I20260809 19:25:31.117034 23255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "69914967a446474b8cffd38632e2f5c6"
format_stamp: "Formatted at 2026-08-09 19:25:31 on dist-test-slave-njxd"
server_key: "2748abaf89a5e1c9cce158ca797c733e"
server_key_iv: "d4fe9fb3dd682a44978eaa7b01023774"
server_key_version: "encryptionkey@0"
I20260809 19:25:31.118384 23255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:25:31.119086 23270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:31.119249 23255 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:31.119345 23255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "69914967a446474b8cffd38632e2f5c6"
format_stamp: "Formatted at 2026-08-09 19:25:31 on dist-test-slave-njxd"
server_key: "2748abaf89a5e1c9cce158ca797c733e"
server_key_iv: "d4fe9fb3dd682a44978eaa7b01023774"
server_key_version: "encryptionkey@0"
I20260809 19:25:31.119422 23255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:31.134763 23255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:31.136502 23255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:31.136656 23255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:31.137187 23255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:31.137250 23255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:31.137288 23255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:31.137308 23255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:25:31 dist-test-slave-njxd krb5kdc[22980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303531, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:31.144941 23255 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:37309
I20260809 19:25:31.145042 23392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:37309 every 8 connection(s)
I20260809 19:25:31.145408 23255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:25:31.147207 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.137553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:44691 (local address 127.21.81.254:40401)
0809 19:25:31.137632 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.137635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.137921 (+ 286us) server_negotiation.cc:407] Connection header received
0809 19:25:31.138616 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.138617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.138627 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.138669 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:31.139581 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.140094 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.141778 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.141997 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.142083 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.144386 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:31.144387 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:31.144388 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:31.144395 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:31.146107 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:31.146456 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:31.146456 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:31.146457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:31.146509 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:31.146674 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:31.146675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:31.146675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:31.146771 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:31.146818 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.147060 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.147134 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 19:25:31.147461 23279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.137552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:40401 (local address 127.21.81.194:44691)
0809 19:25:31.137766 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:31.137817 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:31.138514 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:31.138743 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:31.138748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:31.139129 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:31.139473 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.139481 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.140176 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.140177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.141686 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.141689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.142103 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.142104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.142233 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.142811 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:31.142818 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:31.144289 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:31.146173 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:31.146176 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:31.146178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:31.146386 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:31.146547 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:31.146548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:31.146548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:31.146625 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:31.146809 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:31.146811 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:31.146931 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 19:25:31.147041 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 19:25:31.147204 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":121,"threads_started":1}
I20260809 19:25:31.147931 23393 heartbeater.cc:347] Connected to a master server at 127.21.81.254:40401
I20260809 19:25:31.148008 23393 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:31.148190 23393 heartbeater.cc:511] Master 127.21.81.254:40401 requested a full tablet report, sending...
I20260809 19:25:31.148530 23028 ts_manager.cc:194] Registered new tserver with Master: 69914967a446474b8cffd38632e2f5c6 (127.21.81.194:37309)
I20260809 19:25:31.148890 23028 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:44691
I20260809 19:25:31.154316 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.149008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:43159 (local address 127.21.81.254:40401)
0809 19:25:31.149055 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.149057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.149184 (+ 127us) server_negotiation.cc:407] Connection header received
0809 19:25:31.149649 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.149650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.149660 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.149719 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:31.150000 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.150446 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.151067 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.151237 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.151284 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.152023 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:31.152024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:31.152024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:31.152029 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:31.153387 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:31.153722 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:31.153722 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:31.153723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:31.153775 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:31.153913 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:31.153913 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:31.153914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:31.154031 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:31.154077 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.154175 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.154228 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:25:31.154481 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 23255
I20260809 19:25:31.154578 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:25:31.154340 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.148957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:40401 (local address 127.21.81.194:43159)
0809 19:25:31.149120 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:31.149130 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:31.149187 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:31.149733 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:31.149734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:31.149875 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:31.149939 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.149940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.150524 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.150524 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.151003 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.151004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.151296 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.151297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.151352 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.151692 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:31.151695 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:31.151949 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:31.153453 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:31.153454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:31.153454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:31.153653 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:31.153824 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:31.153825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:31.153825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:31.153876 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:31.154077 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:31.154078 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:31.154112 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 19:25:31.154160 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 19:25:31.154233 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":68,"threads_started":1}
I20260809 19:25:31.154788 21831 external_mini_cluster.cc:1428] Setting key 0d628185a38fcbe3e6cb72e053565914
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:25:31.179807 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:40401
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:31.261453 23402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:31.261729 23402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:31.261771 23402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:31.263060 23402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:31.263118 23402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:31.263154 23402 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:25:31.263208 23402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:25:31.264822 23402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.81.254:40401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:31.265086 23402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:31.265312 23402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:31.270291 23410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:31.270291 23407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:31.270437 23402 server_base.cc:1034] running on GCE node
W20260809 19:25:31.270345 23408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:31.270715 23402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:31.270996 23402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:31.272150 23402 hybrid_clock.cc:648] HybridClock initialized: now 1786303531272143 us; error 35 us; skew 500 ppm
Aug 09 19:25:31 dist-test-slave-njxd krb5kdc[22980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303531, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:31.274557 23402 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:25:31.275036 23402 webserver.cc:460] Webserver started at http://127.21.81.195:38283/ using document root <none> and password file <none>
I20260809 19:25:31.275234 23402 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:31.275276 23402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:31.275362 23402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:31.276327 23402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "88097dbb9dea487cac6331fa15b74a9d"
format_stamp: "Formatted at 2026-08-09 19:25:31 on dist-test-slave-njxd"
server_key: "30ab7285b774b8e715ff9f926f3e8c8d"
server_key_iv: "12fb0514d8d9e628274a2d8cea3d4ec1"
server_key_version: "encryptionkey@0"
I20260809 19:25:31.276612 23402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "88097dbb9dea487cac6331fa15b74a9d"
format_stamp: "Formatted at 2026-08-09 19:25:31 on dist-test-slave-njxd"
server_key: "30ab7285b774b8e715ff9f926f3e8c8d"
server_key_iv: "12fb0514d8d9e628274a2d8cea3d4ec1"
server_key_version: "encryptionkey@0"
I20260809 19:25:31.277873 23402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:25:31.278535 23419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:31.278733 23402 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:31.278831 23402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "88097dbb9dea487cac6331fa15b74a9d"
format_stamp: "Formatted at 2026-08-09 19:25:31 on dist-test-slave-njxd"
server_key: "30ab7285b774b8e715ff9f926f3e8c8d"
server_key_iv: "12fb0514d8d9e628274a2d8cea3d4ec1"
server_key_version: "encryptionkey@0"
I20260809 19:25:31.278908 23402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:31.300016 23402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:31.302140 23402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:31.302299 23402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:31.302955 23402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:31.303009 23402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:31.303040 23402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:31.303057 23402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:25:31 dist-test-slave-njxd krb5kdc[22980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303531, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:31.310626 23402 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:34127
I20260809 19:25:31.310715 23540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:34127 every 8 connection(s)
I20260809 19:25:31.311040 23402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:25:31.313511 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.303515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:47611 (local address 127.21.81.254:40401)
0809 19:25:31.303557 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.303559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.303790 (+ 231us) server_negotiation.cc:407] Connection header received
0809 19:25:31.304459 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.304461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.304473 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.304522 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:31.305101 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.305781 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.306772 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.307141 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.307242 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.310884 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:31.310885 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:31.310885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:31.310893 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:31.312423 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:31.312810 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:31.312811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:31.312811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:31.312861 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:31.313027 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:31.313028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:31.313028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:31.313125 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:31.313181 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.313390 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.313447 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:25:31.313757 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.303517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:40401 (local address 127.21.81.195:47611)
0809 19:25:31.303675 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:31.303709 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:31.304345 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:31.304587 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:31.304590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:31.304835 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:31.305009 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.305014 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.305889 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.305890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.306663 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.306665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.307313 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.307315 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.307476 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.308050 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:31.308062 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:31.310755 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:31.312498 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:31.312502 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:31.312504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:31.312739 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:31.312901 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:31.312901 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:31.312902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:31.312982 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:31.313185 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:31.313190 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:31.313296 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 19:25:31.313370 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 19:25:31.313483 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260809 19:25:31.314262 23541 heartbeater.cc:347] Connected to a master server at 127.21.81.254:40401
I20260809 19:25:31.314343 23541 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:31.314535 23541 heartbeater.cc:511] Master 127.21.81.254:40401 requested a full tablet report, sending...
I20260809 19:25:31.314972 23027 ts_manager.cc:194] Registered new tserver with Master: 88097dbb9dea487cac6331fa15b74a9d (127.21.81.195:34127)
I20260809 19:25:31.315505 23027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:47611
I20260809 19:25:31.315582 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 23402
I20260809 19:25:31.315672 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:25:31.315857 21831 external_mini_cluster.cc:1428] Setting key 1a8158af9d5e92cd3fd5b5b84514a6a7
I20260809 19:25:31.316553 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:25:31.321136 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.315464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:50639 (local address 127.21.81.254:40401)
0809 19:25:31.315508 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.315510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.315783 (+ 273us) server_negotiation.cc:407] Connection header received
0809 19:25:31.315859 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.315860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.315870 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.315918 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:31.316326 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.316856 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.317510 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.317814 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.317882 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.318818 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:31.318819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:31.318819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:31.318824 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:31.320049 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:31.320450 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:31.320450 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:31.320451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:31.320496 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:31.320700 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:31.320700 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:31.320700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:31.320785 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:31.320832 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.321009 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.321078 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:25:31.321252 23548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.315376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:40401 (local address 127.21.81.195:50639)
0809 19:25:31.315695 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:31.315713 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:31.315796 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:31.315959 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:31.315960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:31.316150 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:31.316231 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.316233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.316946 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.316947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.317435 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.317436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.317887 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.317888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.317942 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.318341 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:31.318345 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:31.318737 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:31.320127 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:31.320128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:31.320128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:31.320379 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:31.320555 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:31.320556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:31.320556 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:31.320634 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:31.320849 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:31.320850 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:31.320901 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 19:25:31.320975 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 19:25:31.321094 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":67,"threads_started":1}
Aug 09 19:25:31 dist-test-slave-njxd krb5kdc[22980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303531, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 19:25:31 dist-test-slave-njxd krb5kdc[22980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303531, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:31.340013 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.333673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36108 (local address 127.21.81.254:40401)
0809 19:25:31.333735 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.333737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.333745 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:31.333771 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.333772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.333781 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.333838 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:31.334369 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.334879 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.335616 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.335839 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.335914 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.337982 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:31.337983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:31.337983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:31.337989 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:31.339082 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:31.339382 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:31.339382 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:31.339383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:31.339429 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:31.339584 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:31.339585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:31.339585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:31.339680 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:31.339750 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.339847 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.339901 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:25:31.341677 23027 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:25:31.343410 23027 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:25:31.349876 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.345986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59538 (local address 127.21.81.193:44423)
0809 19:25:31.346175 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.346183 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.346199 (+ 16us) server_negotiation.cc:407] Connection header received
0809 19:25:31.346237 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.346239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.346254 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.346500 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:31.346572 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.347260 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.348890 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.349426 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.349506 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.349561 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.349726 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.349776 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260809 19:25:31.349886 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.345537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:44423 (local address 127.0.0.1:59538)
0809 19:25:31.345882 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:31.345890 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:31.345925 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:31.346352 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:31.346353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:31.346355 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:31.346421 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.346423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.347391 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.347392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.348831 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.348833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.349506 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.349507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.349621 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.349638 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 19:25:31.349707 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 19:25:31.349761 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":34,"thread_start_us":98,"threads_started":1}
I20260809 19:25:31.350067 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.345779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56956 (local address 127.21.81.194:37309)
0809 19:25:31.345960 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.345965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.345975 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:25:31.346013 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.346016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.346030 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.346122 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:31.346367 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.347430 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.348817 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.349490 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.349551 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.349619 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.349926 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.349970 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":87,"threads_started":1}
I20260809 19:25:31.350091 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.345504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:34127 (local address 127.0.0.1:38228)
0809 19:25:31.345818 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:31.345826 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:31.345885 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:31.346208 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:31.346211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:31.346213 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:31.346272 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.346274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.347506 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.347507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.348675 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.348677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.349629 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.349630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.349692 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.349695 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:25:31.349739 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:25:31.349764 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":77,"thread_start_us":86,"threads_started":1}
I20260809 19:25:31.350238 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.345504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:37309 (local address 127.0.0.1:56956)
0809 19:25:31.345691 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:31.345728 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:31.345788 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:31.346252 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:31.346253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:31.346255 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:31.346292 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.346293 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.347580 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.347581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.348747 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.348750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.349654 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.349655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.349728 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.349733 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:25:31.349780 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:25:31.349817 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":19,"thread_start_us":59,"threads_started":1}
I20260809 19:25:31.350281 23563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.345641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.21.81.195:34127)
0809 19:25:31.345994 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.345998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.346006 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:31.346037 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.346039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.346053 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.346130 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:31.346360 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.347430 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.348759 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.349484 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.349551 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.349619 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.349866 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.349905 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":92,"threads_started":1}
I20260809 19:25:31.350641 23176 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3b6ddfc33046fc887f42dd978b5d0b (DEFAULT_TABLE table=test-table [id=e588600304774a51a4b6a812d7492190]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:31.350834 23472 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3b6ddfc33046fc887f42dd978b5d0b (DEFAULT_TABLE table=test-table [id=e588600304774a51a4b6a812d7492190]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:31.350840 23306 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3b6ddfc33046fc887f42dd978b5d0b (DEFAULT_TABLE table=test-table [id=e588600304774a51a4b6a812d7492190]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:31.351028 23176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3b6ddfc33046fc887f42dd978b5d0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:31.351187 23306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3b6ddfc33046fc887f42dd978b5d0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:31.351186 23472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3b6ddfc33046fc887f42dd978b5d0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:31.354571 23567 tablet_bootstrap.cc:492] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d: Bootstrap starting.
I20260809 19:25:31.354676 23566 tablet_bootstrap.cc:492] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72: Bootstrap starting.
I20260809 19:25:31.354915 23568 tablet_bootstrap.cc:492] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6: Bootstrap starting.
I20260809 19:25:31.355353 23566 tablet_bootstrap.cc:654] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:31.355512 23567 tablet_bootstrap.cc:654] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:31.355590 23568 tablet_bootstrap.cc:654] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:31.355762 23566 log.cc:824] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:31.355880 23567 log.cc:824] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:31.355933 23568 log.cc:824] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:31.356594 23566 tablet_bootstrap.cc:492] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72: No bootstrap required, opened a new log
I20260809 19:25:31.356606 23567 tablet_bootstrap.cc:492] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d: No bootstrap required, opened a new log
I20260809 19:25:31.356628 23568 tablet_bootstrap.cc:492] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6: No bootstrap required, opened a new log
I20260809 19:25:31.356683 23567 ts_tablet_manager.cc:1397] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:25:31.356681 23566 ts_tablet_manager.cc:1397] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:25:31.356812 23568 ts_tablet_manager.cc:1397] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:25:31.358685 23566 raft_consensus.cc:348] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } }
I20260809 19:25:31.358834 23566 raft_consensus.cc:374] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:31.358876 23566 raft_consensus.cc:729] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d439dd42f8d44cc8936df6fa9ecf5c72, State: Initialized, Role: FOLLOWER
I20260809 19:25:31.358976 23566 consensus_queue.cc:260] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } }
I20260809 19:25:31.359251 23566 ts_tablet_manager.cc:1428] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:25:31.359263 23246 heartbeater.cc:503] Master 127.21.81.254:40401 was elected leader, sending a full tablet report...
I20260809 19:25:31.359421 23568 raft_consensus.cc:348] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } }
I20260809 19:25:31.359577 23568 raft_consensus.cc:374] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:31.359513 23567 raft_consensus.cc:348] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } }
I20260809 19:25:31.359620 23568 raft_consensus.cc:729] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69914967a446474b8cffd38632e2f5c6, State: Initialized, Role: FOLLOWER
I20260809 19:25:31.359637 23567 raft_consensus.cc:374] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:31.359679 23567 raft_consensus.cc:729] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88097dbb9dea487cac6331fa15b74a9d, State: Initialized, Role: FOLLOWER
I20260809 19:25:31.359735 23568 consensus_queue.cc:260] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } }
I20260809 19:25:31.359807 23567 consensus_queue.cc:260] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } }
I20260809 19:25:31.360004 23568 ts_tablet_manager.cc:1428] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:25:31.360034 23567 ts_tablet_manager.cc:1428] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 19:25:31.360131 23393 heartbeater.cc:503] Master 127.21.81.254:40401 was elected leader, sending a full tablet report...
I20260809 19:25:31.360213 23541 heartbeater.cc:503] Master 127.21.81.254:40401 was elected leader, sending a full tablet report...
W20260809 19:25:31.396171 23394 tablet.cc:2369] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:25:31.485877 23247 tablet.cc:2369] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:25:31.561790 23542 tablet.cc:2369] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:25:31.647799 23575 raft_consensus.cc:482] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:25:31.647935 23575 raft_consensus.cc:504] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } }
I20260809 19:25:31.648407 23575 leader_election.cc:288] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d439dd42f8d44cc8936df6fa9ecf5c72 (127.21.81.193:44423), 69914967a446474b8cffd38632e2f5c6 (127.21.81.194:37309)
I20260809 19:25:31.648648 23574 raft_consensus.cc:482] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:25:31.648837 23574 raft_consensus.cc:504] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } }
I20260809 19:25:31.649637 23574 leader_election.cc:288] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88097dbb9dea487cac6331fa15b74a9d (127.21.81.195:34127), d439dd42f8d44cc8936df6fa9ecf5c72 (127.21.81.193:44423)
I20260809 19:25:31.652514 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.648644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:36635 (local address 127.21.81.193:44423)
0809 19:25:31.648900 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.648903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.648911 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:31.648943 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.648944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.648955 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.649012 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:31.649331 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.650190 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.651595 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.652105 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.652176 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.652187 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.652408 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.652445 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228}
I20260809 19:25:31.652854 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.648637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:44423 (local address 127.21.81.195:36635)
0809 19:25:31.648718 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:31.648731 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:31.648787 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:31.649069 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:31.649070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:31.649072 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:31.649230 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.649233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.650274 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.650275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.651489 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.651492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.652218 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.652219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.652315 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.652318 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:25:31.652365 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:25:31.652772 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":337}
I20260809 19:25:31.653254 23579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.649546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:54231 (local address 127.21.81.193:44423)
0809 19:25:31.649716 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.649718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.649808 (+ 90us) server_negotiation.cc:407] Connection header received
0809 19:25:31.649891 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.649892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.649900 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.649935 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:31.650156 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.650844 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.652201 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.652778 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.652829 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.652837 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.653130 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.653180 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":66,"threads_started":1}
I20260809 19:25:31.653268 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.648711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:54823 (local address 127.21.81.194:37309)
0809 19:25:31.648767 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.648770 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.649015 (+ 245us) server_negotiation.cc:407] Connection header received
0809 19:25:31.649123 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.649125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.649134 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.649216 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:31.649504 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.650243 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.652310 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.652824 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.652874 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.652882 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.653195 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.653226 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 19:25:31.653419 23563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.649631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:60131 (local address 127.21.81.195:34127)
0809 19:25:31.649799 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.649800 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.649809 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:25:31.649829 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.649830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.649838 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.649892 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:31.650059 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.650869 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.652320 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.652833 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.652892 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.652899 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.653321 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.653359 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260809 19:25:31.653373 23196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3b6ddfc33046fc887f42dd978b5d0b" candidate_uuid: "88097dbb9dea487cac6331fa15b74a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" is_pre_election: true
I20260809 19:25:31.653528 23576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.648641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:37309 (local address 127.21.81.195:54823)
0809 19:25:31.648944 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:31.648956 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:31.649056 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:31.649223 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:31.649224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:31.649225 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:31.649403 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.649407 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.650335 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.650335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.652200 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.652203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.653003 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.653005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.653110 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.653114 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:31.653398 (+ 284us) client_negotiation.cc:240] Negotiation successful
0809 19:25:31.653433 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":56,"thread_start_us":136,"threads_started":1}
I20260809 19:25:31.653292 23577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.649373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:34127 (local address 127.21.81.194:60131)
0809 19:25:31.649535 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:31.649545 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:31.649615 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:31.649930 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:31.649931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:31.649934 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:31.649994 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.649996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.650984 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.650985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.652224 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.652227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.652932 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.652933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.653045 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.653049 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:31.653115 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 19:25:31.653162 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":68,"threads_started":1}
I20260809 19:25:31.653538 23196 raft_consensus.cc:2454] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88097dbb9dea487cac6331fa15b74a9d in term 0.
I20260809 19:25:31.653517 23578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.649397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:44423 (local address 127.21.81.194:54231)
0809 19:25:31.649761 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:31.649769 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:31.649826 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:31.650005 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:31.650006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:31.650020 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:31.650086 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.650088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.650946 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.650947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.652103 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.652106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.652900 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.652901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.653028 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.653032 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:31.653327 (+ 295us) client_negotiation.cc:240] Negotiation successful
0809 19:25:31.653367 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":47,"threads_started":1}
I20260809 19:25:31.653909 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3b6ddfc33046fc887f42dd978b5d0b" candidate_uuid: "88097dbb9dea487cac6331fa15b74a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69914967a446474b8cffd38632e2f5c6" is_pre_election: true
I20260809 19:25:31.653909 23473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3b6ddfc33046fc887f42dd978b5d0b" candidate_uuid: "69914967a446474b8cffd38632e2f5c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88097dbb9dea487cac6331fa15b74a9d" is_pre_election: true
I20260809 19:25:31.654007 23473 raft_consensus.cc:2454] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69914967a446474b8cffd38632e2f5c6 in term 0.
I20260809 19:25:31.654007 23345 raft_consensus.cc:2454] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88097dbb9dea487cac6331fa15b74a9d in term 0.
I20260809 19:25:31.654126 23420 leader_election.cc:302] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88097dbb9dea487cac6331fa15b74a9d, d439dd42f8d44cc8936df6fa9ecf5c72; no voters:
I20260809 19:25:31.654165 23196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3b6ddfc33046fc887f42dd978b5d0b" candidate_uuid: "69914967a446474b8cffd38632e2f5c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" is_pre_election: true
I20260809 19:25:31.654225 23196 raft_consensus.cc:2454] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69914967a446474b8cffd38632e2f5c6 in term 0.
I20260809 19:25:31.654259 23575 raft_consensus.cc:2781] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:25:31.654255 23273 leader_election.cc:302] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69914967a446474b8cffd38632e2f5c6, 88097dbb9dea487cac6331fa15b74a9d; no voters:
I20260809 19:25:31.654313 23575 raft_consensus.cc:482] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:25:31.654336 23575 raft_consensus.cc:3037] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:31.654377 23574 raft_consensus.cc:2781] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:25:31.654419 23574 raft_consensus.cc:482] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:25:31.654460 23574 raft_consensus.cc:3037] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:31.655241 23574 raft_consensus.cc:504] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } }
I20260809 19:25:31.655334 23575 raft_consensus.cc:504] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } }
I20260809 19:25:31.655383 23574 leader_election.cc:288] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [CANDIDATE]: Term 1 election: Requested vote from peers 88097dbb9dea487cac6331fa15b74a9d (127.21.81.195:34127), d439dd42f8d44cc8936df6fa9ecf5c72 (127.21.81.193:44423)
I20260809 19:25:31.655467 23575 leader_election.cc:288] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [CANDIDATE]: Term 1 election: Requested vote from peers d439dd42f8d44cc8936df6fa9ecf5c72 (127.21.81.193:44423), 69914967a446474b8cffd38632e2f5c6 (127.21.81.194:37309)
I20260809 19:25:31.655550 23473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3b6ddfc33046fc887f42dd978b5d0b" candidate_uuid: "69914967a446474b8cffd38632e2f5c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88097dbb9dea487cac6331fa15b74a9d"
I20260809 19:25:31.655608 23196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3b6ddfc33046fc887f42dd978b5d0b" candidate_uuid: "69914967a446474b8cffd38632e2f5c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72"
I20260809 19:25:31.655601 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3b6ddfc33046fc887f42dd978b5d0b" candidate_uuid: "88097dbb9dea487cac6331fa15b74a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69914967a446474b8cffd38632e2f5c6"
I20260809 19:25:31.655644 23473 raft_consensus.cc:2379] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 69914967a446474b8cffd38632e2f5c6 in current term 1: Already voted for candidate 88097dbb9dea487cac6331fa15b74a9d in this term.
I20260809 19:25:31.655686 23345 raft_consensus.cc:2379] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 88097dbb9dea487cac6331fa15b74a9d in current term 1: Already voted for candidate 69914967a446474b8cffd38632e2f5c6 in this term.
I20260809 19:25:31.655670 23196 raft_consensus.cc:3037] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:31.655726 23195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3b6ddfc33046fc887f42dd978b5d0b" candidate_uuid: "88097dbb9dea487cac6331fa15b74a9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72"
I20260809 19:25:31.656036 23420 leader_election.cc:302] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88097dbb9dea487cac6331fa15b74a9d; no voters: 69914967a446474b8cffd38632e2f5c6, d439dd42f8d44cc8936df6fa9ecf5c72
I20260809 19:25:31.656126 23575 raft_consensus.cc:2726] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 19:25:31.656649 23196 raft_consensus.cc:2454] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69914967a446474b8cffd38632e2f5c6 in term 1.
I20260809 19:25:31.656821 23271 leader_election.cc:302] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 69914967a446474b8cffd38632e2f5c6, d439dd42f8d44cc8936df6fa9ecf5c72; no voters: 88097dbb9dea487cac6331fa15b74a9d
I20260809 19:25:31.656967 23574 raft_consensus.cc:2781] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:31.657109 23574 raft_consensus.cc:686] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [term 1 LEADER]: Becoming Leader. State: Replica: 69914967a446474b8cffd38632e2f5c6, State: Running, Role: LEADER
I20260809 19:25:31.657224 23574 consensus_queue.cc:237] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } }
I20260809 19:25:31.658072 23028 catalog_manager.cc:5495] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69914967a446474b8cffd38632e2f5c6 (127.21.81.194). New cstate: current_term: 1 leader_uuid: "69914967a446474b8cffd38632e2f5c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } health_report { overall_health: HEALTHY } } }
I20260809 19:25:31.705777 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.703112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53694 (local address 127.21.81.194:37309)
0809 19:25:31.703167 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.703170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.703190 (+ 20us) server_negotiation.cc:407] Connection header received
0809 19:25:31.703308 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.703309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.703337 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.703431 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:31.703635 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.704209 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.704927 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.705130 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.705198 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.705408 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:31.705510 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:31.705608 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.705673 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.705693 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:25:31.707705 23196 raft_consensus.cc:1261] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72 [term 1 FOLLOWER]: Refusing update from remote peer 69914967a446474b8cffd38632e2f5c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:31.707724 23473 raft_consensus.cc:1261] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [term 1 FOLLOWER]: Refusing update from remote peer 69914967a446474b8cffd38632e2f5c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:31.707975 23580 consensus_queue.cc:1035] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:31.708118 23574 consensus_queue.cc:1035] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:31.713938 23028 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:31.714026 23028 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:31.714829 23028 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b: Sending DeleteTablet for 3 replicas of tablet 4e3b6ddfc33046fc887f42dd978b5d0b
I20260809 19:25:31.715161 23472 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3b6ddfc33046fc887f42dd978b5d0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:31 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:38228
I20260809 19:25:31.715197 23176 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3b6ddfc33046fc887f42dd978b5d0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:31 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:59538
I20260809 19:25:31.715400 23306 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3b6ddfc33046fc887f42dd978b5d0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:31 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:56956
I20260809 19:25:31.715708 23594 tablet_replica.cc:330] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d: stopping tablet replica
I20260809 19:25:31.715737 23595 tablet_replica.cc:330] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6: stopping tablet replica
I20260809 19:25:31.715688 23028 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:25:31.715862 23028 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:25:31.715865 23593 tablet_replica.cc:330] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72: stopping tablet replica
I20260809 19:25:31.715955 23593 raft_consensus.cc:2229] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:31.716048 23593 raft_consensus.cc:2258] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:31.716149 23595 raft_consensus.cc:2229] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:25:31.716199 23594 raft_consensus.cc:2229] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:31.716369 23594 raft_consensus.cc:2258] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:31.716626 23593 ts_tablet_manager.cc:1905] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:31.716827 23594 ts_tablet_manager.cc:1905] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:31.717550 23306 tablet_service.cc:1467] Processing CreateTablet for tablet c683e00ddf464ca382510551a257cb05 (DEFAULT_TABLE table=test-table [id=cf4c7a0d1a5043329bd246a125d8358e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:31.717695 23306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c683e00ddf464ca382510551a257cb05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:31.718710 23593 ts_tablet_manager.cc:1918] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:25:31.718864 23593 log.cc:1196] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/wal/wals/4e3b6ddfc33046fc887f42dd978b5d0b
I20260809 19:25:31.718896 23594 ts_tablet_manager.cc:1918] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:25:31.718971 23594 log.cc:1196] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/4e3b6ddfc33046fc887f42dd978b5d0b
I20260809 19:25:31.719210 23593 ts_tablet_manager.cc:1939] T 4e3b6ddfc33046fc887f42dd978b5d0b P d439dd42f8d44cc8936df6fa9ecf5c72: Deleting consensus metadata
I20260809 19:25:31.719262 23594 ts_tablet_manager.cc:1939] T 4e3b6ddfc33046fc887f42dd978b5d0b P 88097dbb9dea487cac6331fa15b74a9d: Deleting consensus metadata
I20260809 19:25:31.719406 23568 tablet_bootstrap.cc:492] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6: Bootstrap starting.
I20260809 19:25:31.719458 23472 tablet_service.cc:1467] Processing CreateTablet for tablet c683e00ddf464ca382510551a257cb05 (DEFAULT_TABLE table=test-table [id=cf4c7a0d1a5043329bd246a125d8358e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:31.719556 23472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c683e00ddf464ca382510551a257cb05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:31.720049 23568 tablet_bootstrap.cc:654] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:31.720877 23014 catalog_manager.cc:4841] TS 88097dbb9dea487cac6331fa15b74a9d (127.21.81.195:34127): tablet 4e3b6ddfc33046fc887f42dd978b5d0b (table test-table [id=e588600304774a51a4b6a812d7492190]) successfully deleted
I20260809 19:25:31.720947 23567 tablet_bootstrap.cc:492] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d: Bootstrap starting.
I20260809 19:25:31.721029 23012 catalog_manager.cc:4841] TS d439dd42f8d44cc8936df6fa9ecf5c72 (127.21.81.193:44423): tablet 4e3b6ddfc33046fc887f42dd978b5d0b (table test-table [id=e588600304774a51a4b6a812d7492190]) successfully deleted
I20260809 19:25:31.721223 23176 tablet_service.cc:1467] Processing CreateTablet for tablet c683e00ddf464ca382510551a257cb05 (DEFAULT_TABLE table=test-table [id=cf4c7a0d1a5043329bd246a125d8358e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:31.721315 23176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c683e00ddf464ca382510551a257cb05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:31.721364 23567 tablet_bootstrap.cc:654] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:31.722590 23567 tablet_bootstrap.cc:492] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d: No bootstrap required, opened a new log
I20260809 19:25:31.722645 23567 ts_tablet_manager.cc:1397] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:25:31.722638 23566 tablet_bootstrap.cc:492] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72: Bootstrap starting.
I20260809 19:25:31.722903 23567 raft_consensus.cc:348] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:31.722950 23567 raft_consensus.cc:374] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:31.722985 23567 raft_consensus.cc:729] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88097dbb9dea487cac6331fa15b74a9d, State: Initialized, Role: FOLLOWER
I20260809 19:25:31.722997 23595 raft_consensus.cc:2258] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:31.723047 23567 consensus_queue.cc:260] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:31.723142 23567 ts_tablet_manager.cc:1428] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:31.723199 23566 tablet_bootstrap.cc:654] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:31.723587 23568 tablet_bootstrap.cc:492] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6: No bootstrap required, opened a new log
I20260809 19:25:31.723632 23568 ts_tablet_manager.cc:1397] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 19:25:31.723778 23568 raft_consensus.cc:348] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:31.723850 23568 raft_consensus.cc:374] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:31.723873 23568 raft_consensus.cc:729] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69914967a446474b8cffd38632e2f5c6, State: Initialized, Role: FOLLOWER
I20260809 19:25:31.723917 23568 consensus_queue.cc:260] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:31.723946 23595 ts_tablet_manager.cc:1905] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:31.724005 23568 ts_tablet_manager.cc:1428] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:31.724479 23566 tablet_bootstrap.cc:492] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72: No bootstrap required, opened a new log
I20260809 19:25:31.724555 23566 ts_tablet_manager.cc:1397] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:25:31.724879 23566 raft_consensus.cc:348] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:31.724951 23566 raft_consensus.cc:374] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:31.724962 23566 raft_consensus.cc:729] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d439dd42f8d44cc8936df6fa9ecf5c72, State: Initialized, Role: FOLLOWER
I20260809 19:25:31.725014 23566 consensus_queue.cc:260] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:31.725092 23566 ts_tablet_manager.cc:1428] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:31.725986 23595 ts_tablet_manager.cc:1918] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:25:31.726068 23595 log.cc:1196] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/4e3b6ddfc33046fc887f42dd978b5d0b
I20260809 19:25:31.726325 23595 ts_tablet_manager.cc:1939] T 4e3b6ddfc33046fc887f42dd978b5d0b P 69914967a446474b8cffd38632e2f5c6: Deleting consensus metadata
I20260809 19:25:31.727026 23015 catalog_manager.cc:4841] TS 69914967a446474b8cffd38632e2f5c6 (127.21.81.194:37309): tablet 4e3b6ddfc33046fc887f42dd978b5d0b (table test-table [id=e588600304774a51a4b6a812d7492190]) successfully deleted
I20260809 19:25:31.829519 23573 raft_consensus.cc:482] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:25:31.829670 23573 raft_consensus.cc:504] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:31.830037 23573 leader_election.cc:288] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69914967a446474b8cffd38632e2f5c6 (127.21.81.194:37309), 88097dbb9dea487cac6331fa15b74a9d (127.21.81.195:34127)
I20260809 19:25:31.834105 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.830441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:39211 (local address 127.21.81.194:37309)
0809 19:25:31.830496 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.830498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.830512 (+ 14us) server_negotiation.cc:407] Connection header received
0809 19:25:31.830548 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.830549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.830574 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.830675 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:31.830998 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.831470 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.833149 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.833740 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.833812 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.833819 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.833994 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.834031 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 19:25:31.834136 23563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.830285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:36803 (local address 127.21.81.195:34127)
0809 19:25:31.830333 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.830335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.830477 (+ 142us) server_negotiation.cc:407] Connection header received
0809 19:25:31.830606 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.830608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.830618 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.830670 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:31.830987 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.831831 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.833203 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.833646 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.833727 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.833745 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.834029 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.834074 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:25:31.834170 23600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.830229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:37309 (local address 127.21.81.193:39211)
0809 19:25:31.830381 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:31.830393 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:31.830470 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:31.830685 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:31.830685 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:31.830687 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:31.830876 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.830877 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.831581 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.831583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.833055 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.833059 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.833812 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.833813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.833905 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.833909 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:31.833980 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:25:31.834031 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":53,"thread_start_us":97,"threads_started":1}
I20260809 19:25:31.834446 23601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.830211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:34127 (local address 127.21.81.193:36803)
0809 19:25:31.830417 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:31.830429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:31.830500 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:31.830715 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:31.830716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:31.830717 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:31.830848 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.830859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.831980 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.831981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.833135 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.833136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.833807 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.833808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.833934 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.833937 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:25:31.834283 (+ 346us) client_negotiation.cc:240] Negotiation successful
0809 19:25:31.834326 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":67,"threads_started":1}
I20260809 19:25:31.834750 23473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c683e00ddf464ca382510551a257cb05" candidate_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88097dbb9dea487cac6331fa15b74a9d" is_pre_election: true
I20260809 19:25:31.834863 23473 raft_consensus.cc:2454] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d439dd42f8d44cc8936df6fa9ecf5c72 in term 0.
I20260809 19:25:31.834880 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c683e00ddf464ca382510551a257cb05" candidate_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69914967a446474b8cffd38632e2f5c6" is_pre_election: true
I20260809 19:25:31.834967 23345 raft_consensus.cc:2454] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d439dd42f8d44cc8936df6fa9ecf5c72 in term 0.
I20260809 19:25:31.835049 23124 leader_election.cc:302] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88097dbb9dea487cac6331fa15b74a9d, d439dd42f8d44cc8936df6fa9ecf5c72; no voters:
I20260809 19:25:31.835209 23573 raft_consensus.cc:2781] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:25:31.835250 23573 raft_consensus.cc:482] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:25:31.835279 23573 raft_consensus.cc:3037] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:31.836417 23573 raft_consensus.cc:504] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:31.836577 23573 leader_election.cc:288] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [CANDIDATE]: Term 1 election: Requested vote from peers 69914967a446474b8cffd38632e2f5c6 (127.21.81.194:37309), 88097dbb9dea487cac6331fa15b74a9d (127.21.81.195:34127)
I20260809 19:25:31.836726 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c683e00ddf464ca382510551a257cb05" candidate_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69914967a446474b8cffd38632e2f5c6"
I20260809 19:25:31.836719 23473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c683e00ddf464ca382510551a257cb05" candidate_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88097dbb9dea487cac6331fa15b74a9d"
I20260809 19:25:31.836793 23345 raft_consensus.cc:3037] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:31.836812 23473 raft_consensus.cc:3037] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:31.837519 23473 raft_consensus.cc:2454] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d439dd42f8d44cc8936df6fa9ecf5c72 in term 1.
I20260809 19:25:31.837540 23345 raft_consensus.cc:2454] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d439dd42f8d44cc8936df6fa9ecf5c72 in term 1.
I20260809 19:25:31.837713 23124 leader_election.cc:302] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 69914967a446474b8cffd38632e2f5c6, 88097dbb9dea487cac6331fa15b74a9d, d439dd42f8d44cc8936df6fa9ecf5c72; no voters:
I20260809 19:25:31.837805 23573 raft_consensus.cc:2781] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:31.837952 23573 raft_consensus.cc:686] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 1 LEADER]: Becoming Leader. State: Replica: d439dd42f8d44cc8936df6fa9ecf5c72, State: Running, Role: LEADER
I20260809 19:25:31.838018 23573 consensus_queue.cc:237] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:31.838709 23028 catalog_manager.cc:5495] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 reported cstate change: term changed from 0 to 1, leader changed from <none> to d439dd42f8d44cc8936df6fa9ecf5c72 (127.21.81.193). New cstate: current_term: 1 leader_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } health_report { overall_health: UNKNOWN } } }
Aug 09 19:25:31 dist-test-slave-njxd krb5kdc[22980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303530, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:31.905603 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.898340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44606 (local address 127.21.81.254:40401)
0809 19:25:31.898480 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:31.898482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:31.898640 (+ 158us) server_negotiation.cc:407] Connection header received
0809 19:25:31.898717 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:31.898718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:31.898728 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:31.898789 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:31.899292 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.899756 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.900403 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.900590 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.900658 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.902862 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:31.902863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:31.902864 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:31.902871 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:31.904404 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:31.904782 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:31.904783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:31.904784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:31.904839 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:31.905029 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:31.905029 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:31.905030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:31.905135 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:31.905213 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:31.905359 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 19:25:31.905464 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260809 19:25:31.905671 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:31.898263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:40401 (local address 127.0.0.1:44606)
0809 19:25:31.898566 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:31.898576 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:31.898658 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:31.898802 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:31.898803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:31.899147 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:31.899196 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.899198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.899829 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.899830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.900319 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:31.900320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:31.900656 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:31.900657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:31.900703 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:31.901227 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:31.901231 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:31.902755 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:31.904489 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:31.904496 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:31.904499 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:31.904697 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:31.904910 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:31.904911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:31.904911 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:31.904968 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:31.905203 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:31.905211 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:31.905254 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 19:25:31.905337 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:25:31.905524 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":86,"thread_start_us":128,"threads_started":1}
I20260809 19:25:31.907421 23028 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:44606:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 19:25:31.907603 23028 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:25:31.909006 23472 tablet_service.cc:1467] Processing CreateTablet for tablet 8a6f2a381ab442a0a5848209e7f6d842 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5cc43e0f70744e8a8110381dbe9c02b1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:25:31.909085 23306 tablet_service.cc:1467] Processing CreateTablet for tablet 8a6f2a381ab442a0a5848209e7f6d842 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5cc43e0f70744e8a8110381dbe9c02b1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:25:31.909101 23176 tablet_service.cc:1467] Processing CreateTablet for tablet 8a6f2a381ab442a0a5848209e7f6d842 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5cc43e0f70744e8a8110381dbe9c02b1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:25:31.909183 23472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a6f2a381ab442a0a5848209e7f6d842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:31.909266 23306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a6f2a381ab442a0a5848209e7f6d842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:31.909264 23176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a6f2a381ab442a0a5848209e7f6d842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:31.910732 23567 tablet_bootstrap.cc:492] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d: Bootstrap starting.
I20260809 19:25:31.910946 23566 tablet_bootstrap.cc:492] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72: Bootstrap starting.
I20260809 19:25:31.911028 23568 tablet_bootstrap.cc:492] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6: Bootstrap starting.
I20260809 19:25:31.911355 23568 tablet_bootstrap.cc:654] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:31.911358 23567 tablet_bootstrap.cc:654] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:31.911689 23566 tablet_bootstrap.cc:654] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:31.911993 23568 tablet_bootstrap.cc:492] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6: No bootstrap required, opened a new log
I20260809 19:25:31.912045 23568 ts_tablet_manager.cc:1397] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:31.912165 23568 raft_consensus.cc:348] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:31.912245 23568 raft_consensus.cc:374] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:31.912271 23568 raft_consensus.cc:729] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69914967a446474b8cffd38632e2f5c6, State: Initialized, Role: FOLLOWER
I20260809 19:25:31.912322 23568 consensus_queue.cc:260] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:31.912418 23568 ts_tablet_manager.cc:1428] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:31.912439 23580 tablet_replica.cc:439] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } }
I20260809 19:25:31.912510 23580 tablet_replica.cc:442] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:25:31.912547 23567 tablet_bootstrap.cc:492] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d: No bootstrap required, opened a new log
I20260809 19:25:31.912593 23567 ts_tablet_manager.cc:1397] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:25:31.912665 23566 tablet_bootstrap.cc:492] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72: No bootstrap required, opened a new log
I20260809 19:25:31.912748 23566 ts_tablet_manager.cc:1397] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:25:31.912959 23567 raft_consensus.cc:348] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:31.913002 23566 raft_consensus.cc:348] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:31.913069 23567 raft_consensus.cc:374] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:31.913077 23566 raft_consensus.cc:374] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:31.913096 23567 raft_consensus.cc:729] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88097dbb9dea487cac6331fa15b74a9d, State: Initialized, Role: FOLLOWER
I20260809 19:25:31.913101 23566 raft_consensus.cc:729] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d439dd42f8d44cc8936df6fa9ecf5c72, State: Initialized, Role: FOLLOWER
I20260809 19:25:31.913161 23567 consensus_queue.cc:260] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:31.913157 23566 consensus_queue.cc:260] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:31.913250 23567 ts_tablet_manager.cc:1428] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:25:31.913264 23566 ts_tablet_manager.cc:1428] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:31.913307 23573 tablet_replica.cc:439] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } }
I20260809 19:25:31.913394 23573 tablet_replica.cc:442] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:25:31.913362 23575 tablet_replica.cc:439] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } }
I20260809 19:25:31.913439 23575 tablet_replica.cc:442] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:25:32.196039 23573 raft_consensus.cc:482] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:25:32.196151 23573 raft_consensus.cc:504] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:32.196378 23573 leader_election.cc:288] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69914967a446474b8cffd38632e2f5c6 (127.21.81.194:37309), 88097dbb9dea487cac6331fa15b74a9d (127.21.81.195:34127)
I20260809 19:25:32.196604 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a6f2a381ab442a0a5848209e7f6d842" candidate_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69914967a446474b8cffd38632e2f5c6" is_pre_election: true
I20260809 19:25:32.196614 23473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a6f2a381ab442a0a5848209e7f6d842" candidate_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88097dbb9dea487cac6331fa15b74a9d" is_pre_election: true
I20260809 19:25:32.196722 23345 raft_consensus.cc:2454] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d439dd42f8d44cc8936df6fa9ecf5c72 in term 0.
I20260809 19:25:32.196717 23473 raft_consensus.cc:2454] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d439dd42f8d44cc8936df6fa9ecf5c72 in term 0.
I20260809 19:25:32.196897 23124 leader_election.cc:302] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88097dbb9dea487cac6331fa15b74a9d, d439dd42f8d44cc8936df6fa9ecf5c72; no voters:
I20260809 19:25:32.197022 23573 raft_consensus.cc:2781] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:25:32.197079 23573 raft_consensus.cc:482] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:25:32.197110 23573 raft_consensus.cc:3037] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:32.197923 23573 raft_consensus.cc:504] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:32.198107 23573 leader_election.cc:288] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [CANDIDATE]: Term 1 election: Requested vote from peers 69914967a446474b8cffd38632e2f5c6 (127.21.81.194:37309), 88097dbb9dea487cac6331fa15b74a9d (127.21.81.195:34127)
I20260809 19:25:32.198225 23345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a6f2a381ab442a0a5848209e7f6d842" candidate_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69914967a446474b8cffd38632e2f5c6"
I20260809 19:25:32.198304 23345 raft_consensus.cc:3037] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:32.198324 23473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a6f2a381ab442a0a5848209e7f6d842" candidate_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88097dbb9dea487cac6331fa15b74a9d"
I20260809 19:25:32.198429 23473 raft_consensus.cc:3037] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:32.199030 23345 raft_consensus.cc:2454] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d439dd42f8d44cc8936df6fa9ecf5c72 in term 1.
I20260809 19:25:32.199200 23473 raft_consensus.cc:2454] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d439dd42f8d44cc8936df6fa9ecf5c72 in term 1.
I20260809 19:25:32.199237 23126 leader_election.cc:302] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69914967a446474b8cffd38632e2f5c6, d439dd42f8d44cc8936df6fa9ecf5c72; no voters:
I20260809 19:25:32.199357 23573 raft_consensus.cc:2781] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:32.199437 23573 raft_consensus.cc:686] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 1 LEADER]: Becoming Leader. State: Replica: d439dd42f8d44cc8936df6fa9ecf5c72, State: Running, Role: LEADER
I20260809 19:25:32.199487 23573 consensus_queue.cc:237] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } }
I20260809 19:25:32.199827 23602 tablet_replica.cc:439] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72: TxnStatusTablet state changed. Reason: New leader d439dd42f8d44cc8936df6fa9ecf5c72. Latest consensus state: current_term: 1 leader_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } }
I20260809 19:25:32.199927 23602 tablet_replica.cc:442] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:25:32.200096 23616 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 19:25:32.200239 23616 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 19:25:32.200176 23028 catalog_manager.cc:5495] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 reported cstate change: term changed from 0 to 1, leader changed from <none> to d439dd42f8d44cc8936df6fa9ecf5c72 (127.21.81.193). New cstate: current_term: 1 leader_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } health_report { overall_health: UNKNOWN } } }
I20260809 19:25:32.261106 23602 consensus_queue.cc:1035] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:32.263212 23573 consensus_queue.cc:1035] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:32.266430 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:32.262618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:44423 (local address 127.0.0.1:35738)
0809 19:25:32.262686 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:32.262699 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:32.262749 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:32.263666 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:32.263668 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:32.263669 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:25:32.263733 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:32.263736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:32.264780 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:32.264782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:32.265368 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:32.265370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:32.265844 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:32.265845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:32.265923 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:32.265944 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:25:32.266277 (+ 333us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:25:32.266287 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 19:25:32.266324 (+ 37us) client_negotiation.cc:240] Negotiation successful
0809 19:25:32.266369 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 19:25:32.266450 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:32.262519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35738 (local address 127.21.81.193:44423)
0809 19:25:32.263464 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:32.263466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:32.263474 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:32.263518 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:32.263519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:32.263538 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:32.263822 (+ 284us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:32.263888 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:32.264659 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:32.265448 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:32.265763 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:32.265831 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:32.266075 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:32.266209 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:32.266292 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:32.266343 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 19:25:32.266367 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":55,"threads_started":1}
I20260809 19:25:32.267515 23473 raft_consensus.cc:1261] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d [term 1 FOLLOWER]: Refusing update from remote peer d439dd42f8d44cc8936df6fa9ecf5c72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:32.267516 23345 raft_consensus.cc:1261] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6 [term 1 FOLLOWER]: Refusing update from remote peer d439dd42f8d44cc8936df6fa9ecf5c72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:32.267765 23602 consensus_queue.cc:1035] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:32.268047 23602 consensus_queue.cc:1035] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:32.268335 23580 tablet_replica.cc:439] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6: TxnStatusTablet state changed. Reason: New leader d439dd42f8d44cc8936df6fa9ecf5c72. Latest consensus state: current_term: 1 leader_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } }
I20260809 19:25:32.268437 23580 tablet_replica.cc:442] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:25:32.268901 23575 tablet_replica.cc:439] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d: TxnStatusTablet state changed. Reason: New leader d439dd42f8d44cc8936df6fa9ecf5c72. Latest consensus state: current_term: 1 leader_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } }
I20260809 19:25:32.269009 23575 tablet_replica.cc:442] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:25:32.268975 23602 tablet_replica.cc:439] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } }
I20260809 19:25:32.269047 23602 tablet_replica.cc:442] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:25:32.269496 23580 tablet_replica.cc:439] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } }
I20260809 19:25:32.269624 23580 tablet_replica.cc:442] T 8a6f2a381ab442a0a5848209e7f6d842 P 69914967a446474b8cffd38632e2f5c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:25:32.269614 23620 mvcc.cc:205] Tried to move back new op lower bound from 7316699268166148096 to 7316699267889491968. Current Snapshot: MvccSnapshot[applied={T|T < 7316699268166148096 or (T in {7316699268166148096})}]
I20260809 19:25:32.270143 23622 mvcc.cc:205] Tried to move back new op lower bound from 7316699268166148096 to 7316699267889491968. Current Snapshot: MvccSnapshot[applied={T|T < 7316699268166148096 or (T in {7316699268166148096})}]
I20260809 19:25:32.270808 23573 tablet_replica.cc:439] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } }
I20260809 19:25:32.270929 23573 tablet_replica.cc:442] T 8a6f2a381ab442a0a5848209e7f6d842 P d439dd42f8d44cc8936df6fa9ecf5c72: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:25:32.271292 23624 mvcc.cc:205] Tried to move back new op lower bound from 7316699268166148096 to 7316699267889491968. Current Snapshot: MvccSnapshot[applied={T|T < 7316699268166148096}]
I20260809 19:25:32.271404 23575 tablet_replica.cc:439] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d439dd42f8d44cc8936df6fa9ecf5c72" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44423 } } peers { permanent_uuid: "69914967a446474b8cffd38632e2f5c6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 37309 } } peers { permanent_uuid: "88097dbb9dea487cac6331fa15b74a9d" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34127 } } }
I20260809 19:25:32.271479 23575 tablet_replica.cc:442] T 8a6f2a381ab442a0a5848209e7f6d842 P 88097dbb9dea487cac6331fa15b74a9d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:25:32.276574 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:32.274012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35750 (local address 127.21.81.193:44423)
0809 19:25:32.274077 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:32.274079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:32.274141 (+ 62us) server_negotiation.cc:407] Connection header received
0809 19:25:32.274198 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:32.274199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:32.274210 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:32.274277 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:32.274378 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:32.275090 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:32.275854 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:32.276124 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:32.276225 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:32.276319 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:32.276353 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:32.276401 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:32.276499 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 19:25:32.276522 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 19:25:32.282038 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:32.279356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:50807 (local address 127.21.81.193:44423)
0809 19:25:32.279398 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:32.279400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:32.279538 (+ 138us) server_negotiation.cc:407] Connection header received
0809 19:25:32.279566 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:32.279567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:32.279576 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:32.279639 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:32.279761 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:32.280409 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:32.281105 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:32.281417 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:32.281494 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:32.281707 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:32.281738 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:32.281801 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:32.281921 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 19:25:32.281944 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":23}
I20260809 19:25:32.282213 23636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:32.279299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:44423 (local address 127.21.81.193:50807)
0809 19:25:32.279458 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:32.279468 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:32.279526 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:32.279647 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:32.279648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:32.279649 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:25:32.279698 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:32.279699 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:32.280487 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:32.280488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:32.281021 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:32.281023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:32.281495 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:32.281496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:32.281544 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:32.281558 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:25:32.281807 (+ 249us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:25:32.281808 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:25:32.281897 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 19:25:32.281945 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260809 19:25:32.296088 23028 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:32.296216 23028 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:32.296928 23028 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f2982bc21a54566952402ab85c2e08b: Sending DeleteTablet for 3 replicas of tablet c683e00ddf464ca382510551a257cb05
I20260809 19:25:32.297214 23306 tablet_service.cc:1514] Processing DeleteTablet for tablet c683e00ddf464ca382510551a257cb05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:32 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:56956
I20260809 19:25:32.297250 23472 tablet_service.cc:1514] Processing DeleteTablet for tablet c683e00ddf464ca382510551a257cb05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:32 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:38228
I20260809 19:25:32.298120 23175 tablet_service.cc:1514] Processing DeleteTablet for tablet c683e00ddf464ca382510551a257cb05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:32 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:59538
I20260809 19:25:32.298472 23643 tablet_replica.cc:330] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72: stopping tablet replica
I20260809 19:25:32.298547 23643 raft_consensus.cc:2229] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:25:32.298628 23643 raft_consensus.cc:2258] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:32.298882 23643 ts_tablet_manager.cc:1905] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:32.301074 23643 ts_tablet_manager.cc:1918] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:25:32.301146 23643 log.cc:1196] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-0/wal/wals/c683e00ddf464ca382510551a257cb05
I20260809 19:25:32.301456 23643 ts_tablet_manager.cc:1939] T c683e00ddf464ca382510551a257cb05 P d439dd42f8d44cc8936df6fa9ecf5c72: Deleting consensus metadata
I20260809 19:25:32.301837 23012 catalog_manager.cc:4841] TS d439dd42f8d44cc8936df6fa9ecf5c72 (127.21.81.193:44423): tablet c683e00ddf464ca382510551a257cb05 (table test-table [id=cf4c7a0d1a5043329bd246a125d8358e]) successfully deleted
Aug 09 19:25:32 dist-test-slave-njxd krb5kdc[22980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303531, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.81.193@KRBTEST.COM
I20260809 19:25:32.305760 23639 tablet_replica.cc:330] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d: stopping tablet replica
I20260809 19:25:32.305848 23639 raft_consensus.cc:2229] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:32.305963 23639 raft_consensus.cc:2258] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:32.306177 23640 tablet_replica.cc:330] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6: stopping tablet replica
I20260809 19:25:32.306306 23640 raft_consensus.cc:2229] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:32.306346 23639 ts_tablet_manager.cc:1905] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:32.306681 23640 raft_consensus.cc:2258] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:32.307029 23640 ts_tablet_manager.cc:1905] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:32.308615 23639 ts_tablet_manager.cc:1918] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:25:32.308734 23639 log.cc:1196] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/c683e00ddf464ca382510551a257cb05
I20260809 19:25:32.309010 23640 ts_tablet_manager.cc:1918] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:25:32.309034 23639 ts_tablet_manager.cc:1939] T c683e00ddf464ca382510551a257cb05 P 88097dbb9dea487cac6331fa15b74a9d: Deleting consensus metadata
I20260809 19:25:32.309063 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:32.298367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35760 (local address 127.21.81.193:44423)
0809 19:25:32.298406 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:32.298408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:32.298823 (+ 415us) server_negotiation.cc:407] Connection header received
0809 19:25:32.298891 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:32.298892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:32.298903 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:32.298985 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:32.299844 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:32.300531 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:32.301781 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:32.302100 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:32.302182 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:32.305335 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:32.305344 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:32.305345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:32.305352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:32.307503 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:32.307941 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:32.307944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:32.307944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:32.308027 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:32.308264 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:32.308265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:32.308265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:32.308444 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:32.308581 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:32.308824 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 19:25:32.308922 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:25:32.309365 23640 log.cc:1196] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/c683e00ddf464ca382510551a257cb05
I20260809 19:25:32.309437 23014 catalog_manager.cc:4841] TS 88097dbb9dea487cac6331fa15b74a9d (127.21.81.195:34127): tablet c683e00ddf464ca382510551a257cb05 (table test-table [id=cf4c7a0d1a5043329bd246a125d8358e]) successfully deleted
W20260809 19:25:32.309698 23240 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35760
I20260809 19:25:32.309734 23640 ts_tablet_manager.cc:1939] T c683e00ddf464ca382510551a257cb05 P 69914967a446474b8cffd38632e2f5c6: Deleting consensus metadata
I20260809 19:25:32.310084 23015 catalog_manager.cc:4841] TS 69914967a446474b8cffd38632e2f5c6 (127.21.81.194:37309): tablet c683e00ddf464ca382510551a257cb05 (table test-table [id=cf4c7a0d1a5043329bd246a125d8358e]) successfully deleted
I20260809 19:25:32.320868 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:32.310644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44612 (local address 127.21.81.254:40401)
0809 19:25:32.313480 (+ 2836us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:32.313482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:32.313629 (+ 147us) server_negotiation.cc:407] Connection header received
0809 19:25:32.313718 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:32.313719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:32.313734 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:32.313807 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:32.314535 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:32.315434 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:32.316103 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:32.316489 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:32.316568 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:32.317993 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:32.317994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:32.317995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:32.318002 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:32.319657 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:32.319977 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:32.319978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:32.319979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:32.320047 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:32.320231 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:32.320232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:32.320232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:32.320431 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:32.320582 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:32.320638 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 19:25:32.320755 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2811}
W20260809 19:25:32.321130 23028 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44612
I20260809 19:25:32.321856 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 23105
I20260809 19:25:32.331831 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 23255
I20260809 19:25:32.339119 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 23402
I20260809 19:25:32.344889 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 22995
2026-08-09T19:25:32Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2874 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:32 dist-test-slave-njxd krb5kdc[23648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:32 dist-test-slave-njxd krb5kdc[23648](info): set up 2 sockets
Aug 09 19:25:32 dist-test-slave-njxd krb5kdc[23648](info): commencing operation
krb5kdc: starting...
W20260809 19:25:33.401192 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:25:33 dist-test-slave-njxd krb5kdc[23648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:25:33.522763 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:44739
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:39449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:44739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:33.603183 23661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:33.603386 23661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:33.603410 23661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:33.604553 23661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:33.604626 23661 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 19:25:33.604640 23661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:33.604681 23661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:33.604712 23661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:33.604729 23661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:33.606218 23661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:39449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:44739
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:44739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:33.606489 23661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:33.606734 23661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:33.611846 23667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:33.611913 23670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:33.611932 23668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:33.612020 23661 server_base.cc:1034] running on GCE node
I20260809 19:25:33.612421 23661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:33.612733 23661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:33.613871 23661 hybrid_clock.cc:648] HybridClock initialized: now 1786303533613852 us; error 35 us; skew 500 ppm
Aug 09 19:25:33 dist-test-slave-njxd krb5kdc[23648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303533, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:33.616276 23661 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:25:33.616758 23661 webserver.cc:460] Webserver started at http://127.21.81.254:41587/ using document root <none> and password file <none>
I20260809 19:25:33.616977 23661 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:33.617045 23661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:33.617206 23661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:33.618317 23661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "85447c7266364b839cdcba820608f862"
format_stamp: "Formatted at 2026-08-09 19:25:33 on dist-test-slave-njxd"
server_key: "758cd01e34de760dec59f84092738fbd"
server_key_iv: "c4ca4cfc242d52bb2f28b6e86fd95eb9"
server_key_version: "encryptionkey@0"
I20260809 19:25:33.618675 23661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "85447c7266364b839cdcba820608f862"
format_stamp: "Formatted at 2026-08-09 19:25:33 on dist-test-slave-njxd"
server_key: "758cd01e34de760dec59f84092738fbd"
server_key_iv: "c4ca4cfc242d52bb2f28b6e86fd95eb9"
server_key_version: "encryptionkey@0"
I20260809 19:25:33.619958 23661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:25:33.620664 23677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:33.620836 23661 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:33.620940 23661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "85447c7266364b839cdcba820608f862"
format_stamp: "Formatted at 2026-08-09 19:25:33 on dist-test-slave-njxd"
server_key: "758cd01e34de760dec59f84092738fbd"
server_key_iv: "c4ca4cfc242d52bb2f28b6e86fd95eb9"
server_key_version: "encryptionkey@0"
I20260809 19:25:33.621012 23661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:33.632333 23661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:33.634049 23661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:33.634204 23661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:33.639133 23661 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:44739
I20260809 19:25:33.639195 23742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:44739 every 8 connection(s)
I20260809 19:25:33.639493 23661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:33.640107 23744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:33.642367 23744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862: Bootstrap starting.
I20260809 19:25:33.642974 23744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:33.643266 23744 log.cc:824] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:33.643977 23744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862: No bootstrap required, opened a new log
I20260809 19:25:33.646133 23744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85447c7266364b839cdcba820608f862" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 44739 } }
I20260809 19:25:33.646268 23744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:33.646291 23744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85447c7266364b839cdcba820608f862, State: Initialized, Role: FOLLOWER
I20260809 19:25:33.646449 23744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85447c7266364b839cdcba820608f862" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 44739 } }
I20260809 19:25:33.646510 23744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:33.646531 23744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:33.646598 23744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:33.647094 23744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85447c7266364b839cdcba820608f862" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 44739 } }
I20260809 19:25:33.647200 23744 leader_election.cc:302] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85447c7266364b839cdcba820608f862; no voters:
I20260809 19:25:33.647373 23744 leader_election.cc:288] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:33.647419 23747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:33.647603 23747 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [term 1 LEADER]: Becoming Leader. State: Replica: 85447c7266364b839cdcba820608f862, State: Running, Role: LEADER
I20260809 19:25:33.647650 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 23661
I20260809 19:25:33.647737 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:33.647660 23744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:33.647817 23747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85447c7266364b839cdcba820608f862" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 44739 } }
I20260809 19:25:33.647907 21831 external_mini_cluster.cc:1428] Setting key 5fa6fa341ef45c27c673d26ab859a597
I20260809 19:25:33.648626 23749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85447c7266364b839cdcba820608f862" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85447c7266364b839cdcba820608f862" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 44739 } } }
I20260809 19:25:33.648663 23750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85447c7266364b839cdcba820608f862. Latest consensus state: current_term: 1 leader_uuid: "85447c7266364b839cdcba820608f862" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85447c7266364b839cdcba820608f862" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 44739 } } }
I20260809 19:25:33.648762 23749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:33.648768 23750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:33.649036 23761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:33.649600 23761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:33.651189 23761 catalog_manager.cc:1347] Generated new cluster ID: 000aea43a90e4c8db0ad3b4f67ed59a1
I20260809 19:25:33.651266 23761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:25:33 dist-test-slave-njxd krb5kdc[23648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:33.658613 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:33.648355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50034 (local address 127.21.81.254:44739)
0809 19:25:33.649242 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:33.649249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:33.649273 (+ 24us) server_negotiation.cc:407] Connection header received
0809 19:25:33.649359 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:33.649364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:33.649426 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:33.649589 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:33.650259 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:33.651508 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:33.652297 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:33.652721 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:33.652847 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:33.655195 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:33.655199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:33.655199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:33.655210 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:33.657128 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:33.657476 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:33.657478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:33.657478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:33.657607 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:33.657798 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:33.657799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:33.657800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:33.658013 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:33.658175 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:33.658277 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 19:25:33.658416 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":832,"thread_start_us":72,"threads_started":1}
I20260809 19:25:33.685417 23761 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:33.686131 23761 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:33.699110 23761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862: Generated new TSK 0
I20260809 19:25:33.699323 23761 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:25:33.726334 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:44739
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:39449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:33.806419 23771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:33.806648 23771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:33.806689 23771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:33.807782 23771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:33.807849 23771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:33.807886 23771 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 19:25:33.807935 23771 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:25:33.807974 23771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:25:33.809453 23771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:39449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.81.254:44739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:33.809754 23771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:33.809983 23771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:33.814918 23777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:33.814949 23771 server_base.cc:1034] running on GCE node
W20260809 19:25:33.814950 23779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:33.814920 23776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:33.815349 23771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:33.815598 23771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:33.816754 23771 hybrid_clock.cc:648] HybridClock initialized: now 1786303533816746 us; error 43 us; skew 500 ppm
Aug 09 19:25:33 dist-test-slave-njxd krb5kdc[23648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303533, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:33.819062 23771 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:25:33.819504 23771 webserver.cc:460] Webserver started at http://127.21.81.193:36385/ using document root <none> and password file <none>
I20260809 19:25:33.819723 23771 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:33.819787 23771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:33.819932 23771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:33.820928 23771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "ecbb40c4b84f4804bfd5d54d340dfa05"
format_stamp: "Formatted at 2026-08-09 19:25:33 on dist-test-slave-njxd"
server_key: "529dcc5acdffd2ed58ea0afe87a1358c"
server_key_iv: "30e573056b8e168d5a98d4c7ffb818dc"
server_key_version: "encryptionkey@0"
I20260809 19:25:33.821296 23771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "ecbb40c4b84f4804bfd5d54d340dfa05"
format_stamp: "Formatted at 2026-08-09 19:25:33 on dist-test-slave-njxd"
server_key: "529dcc5acdffd2ed58ea0afe87a1358c"
server_key_iv: "30e573056b8e168d5a98d4c7ffb818dc"
server_key_version: "encryptionkey@0"
I20260809 19:25:33.822692 23771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:25:33.823491 23787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:33.823720 23771 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:33.823817 23771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "ecbb40c4b84f4804bfd5d54d340dfa05"
format_stamp: "Formatted at 2026-08-09 19:25:33 on dist-test-slave-njxd"
server_key: "529dcc5acdffd2ed58ea0afe87a1358c"
server_key_iv: "30e573056b8e168d5a98d4c7ffb818dc"
server_key_version: "encryptionkey@0"
I20260809 19:25:33.823887 23771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:33.840435 23771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:33.842610 23771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:33.842773 23771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:33.843269 23771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:33.843304 23771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:33.843328 23771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:33.843379 23771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:33.850368 23771 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:41263
Aug 09 19:25:33 dist-test-slave-njxd krb5kdc[23648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303533, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:33.850440 23905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:41263 every 8 connection(s)
I20260809 19:25:33.850754 23771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:25:33.851620 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 23771
I20260809 19:25:33.851701 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:25:33.851925 21831 external_mini_cluster.cc:1428] Setting key 78b7e670e7d5f8c772c020d4ad8b1fa6
I20260809 19:25:33.853909 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:33.843776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:39041 (local address 127.21.81.254:44739)
0809 19:25:33.843854 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:33.843857 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:33.844000 (+ 143us) server_negotiation.cc:407] Connection header received
0809 19:25:33.844452 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:33.844454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:33.844473 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:33.844548 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:33.845307 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:33.846212 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:33.847785 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:33.848159 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:33.848231 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:33.850969 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:33.850970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:33.850971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:33.850980 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:33.852439 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:33.852928 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:33.852929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:33.852929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:33.852994 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:33.853276 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:33.853276 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:33.853277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:33.853372 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:33.853422 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:33.853779 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 19:25:33.853848 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 19:25:33.854189 23796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:33.843705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:44739 (local address 127.21.81.193:39041)
0809 19:25:33.843876 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:33.843913 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:33.844360 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:33.844610 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:33.844613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:33.844936 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:33.845195 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:33.845202 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:33.846311 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:33.846312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:33.847687 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:33.847690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:33.848255 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:33.848256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:33.848385 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:33.848964 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:33.848983 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:33.850861 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:33.852500 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:33.852508 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:33.852510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:33.852850 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:33.853071 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:33.853072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:33.853072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:33.853212 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:33.853435 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:33.853441 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:33.853644 (+ 203us) client_negotiation.cc:769] Sending connection context
0809 19:25:33.853762 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 19:25:33.853934 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260809 19:25:33.854719 23906 heartbeater.cc:347] Connected to a master server at 127.21.81.254:44739
I20260809 19:25:33.854852 23906 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:33.855052 23906 heartbeater.cc:511] Master 127.21.81.254:44739 requested a full tablet report, sending...
I20260809 19:25:33.855679 23694 ts_manager.cc:194] Registered new tserver with Master: ecbb40c4b84f4804bfd5d54d340dfa05 (127.21.81.193:41263)
I20260809 19:25:33.856271 23694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:39041
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
I20260809 19:25:33.863157 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:33.855762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:55005 (local address 127.21.81.254:44739)
0809 19:25:33.855796 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:33.855798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:33.857765 (+ 1967us) server_negotiation.cc:407] Connection header received
0809 19:25:33.857824 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:33.857825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:33.857833 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:33.857881 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:33.858262 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:33.858726 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:33.859698 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:33.859865 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:33.859911 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:33.860845 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:33.860846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:33.860846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:33.860852 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:33.862242 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:33.862586 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:33.862587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:33.862587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:33.862637 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:33.862806 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:33.862806 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:33.862806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:33.862906 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:33.862953 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:33.863049 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 19:25:33.863105 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:25:33.863161 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:33.855706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:44739 (local address 127.21.81.193:55005)
0809 19:25:33.857688 (+ 1982us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:33.857702 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:33.857767 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:33.857906 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:33.857907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:33.858126 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:33.858189 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:33.858191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:33.858820 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:33.858821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:33.859604 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:33.859607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:33.859934 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:33.859935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:33.860012 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:33.860361 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:33.860366 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:33.860750 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:33.862312 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:33.862313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:33.862313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:33.862513 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:33.862700 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:33.862700 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:33.862700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:33.862753 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:33.862954 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:33.862954 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:33.862988 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 19:25:33.863035 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:25:33.863105 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1943,"thread_start_us":80,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:25:33.879343 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:44739
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:39449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:33.961213 23916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:33.961405 23916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:33.961428 23916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:33.962585 23916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:33.962632 23916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:33.962652 23916 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 19:25:33.962664 23916 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:25:33.962680 23916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:25:33.964104 23916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:39449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.81.254:44739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:33.964304 23916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:33.964475 23916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:33.969194 23922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:33.969288 23921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:33.969218 23924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:33.969626 23916 server_base.cc:1034] running on GCE node
I20260809 19:25:33.969846 23916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:33.970105 23916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:33.971262 23916 hybrid_clock.cc:648] HybridClock initialized: now 1786303533971259 us; error 38 us; skew 500 ppm
Aug 09 19:25:33 dist-test-slave-njxd krb5kdc[23648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303533, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:33.973814 23916 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:25:33.974205 23916 webserver.cc:460] Webserver started at http://127.21.81.194:43913/ using document root <none> and password file <none>
I20260809 19:25:33.974399 23916 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:33.974439 23916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:33.974527 23916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:33.975489 23916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "61d635e1443c497dba9326ab5b7de3a6"
format_stamp: "Formatted at 2026-08-09 19:25:33 on dist-test-slave-njxd"
server_key: "32c5108ff7d59c222b8c6d648f9a6d8e"
server_key_iv: "0e5b811197981f9556f56ef074eaf549"
server_key_version: "encryptionkey@0"
I20260809 19:25:33.975771 23916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "61d635e1443c497dba9326ab5b7de3a6"
format_stamp: "Formatted at 2026-08-09 19:25:33 on dist-test-slave-njxd"
server_key: "32c5108ff7d59c222b8c6d648f9a6d8e"
server_key_iv: "0e5b811197981f9556f56ef074eaf549"
server_key_version: "encryptionkey@0"
I20260809 19:25:33.976984 23916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:25:33.977671 23932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:33.977833 23916 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:33.977919 23916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "61d635e1443c497dba9326ab5b7de3a6"
format_stamp: "Formatted at 2026-08-09 19:25:33 on dist-test-slave-njxd"
server_key: "32c5108ff7d59c222b8c6d648f9a6d8e"
server_key_iv: "0e5b811197981f9556f56ef074eaf549"
server_key_version: "encryptionkey@0"
I20260809 19:25:33.977988 23916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:34.000391 23916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:34.002348 23916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:34.002503 23916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:34.003048 23916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:34.003085 23916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:34.003108 23916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:34.003151 23916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:34.010151 23916 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:41575
I20260809 19:25:34.010232 24053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:41575 every 8 connection(s)
Aug 09 19:25:34 dist-test-slave-njxd krb5kdc[23648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303533, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:34.010533 23916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:25:34.013325 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.003585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:56943 (local address 127.21.81.254:44739)
0809 19:25:34.003639 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.003641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.003841 (+ 200us) server_negotiation.cc:407] Connection header received
0809 19:25:34.004398 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.004399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.004413 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.004471 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:34.005280 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.006104 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.007770 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.008095 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.008198 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.010807 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:34.010808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:34.010809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:34.010817 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:34.012212 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:34.012565 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:34.012565 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:34.012566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:34.012618 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:34.012834 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:34.012834 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:34.012835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:34.012945 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:34.013003 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.013206 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.013267 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:25:34.013513 23942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.003569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:44739 (local address 127.21.81.194:56943)
0809 19:25:34.003722 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.003761 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.004289 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.004537 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.004541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.004902 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:34.005191 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.005198 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.006193 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.006194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.007663 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.007666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.008195 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.008195 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.008295 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.008839 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:34.008850 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:34.010702 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:34.012281 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:34.012287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:34.012288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:34.012500 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:34.012678 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:34.012678 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:34.012678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:34.012782 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:34.013008 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:34.013012 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:34.013109 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.013181 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.013287 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
I20260809 19:25:34.013993 24054 heartbeater.cc:347] Connected to a master server at 127.21.81.254:44739
I20260809 19:25:34.014068 24054 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:34.014251 24054 heartbeater.cc:511] Master 127.21.81.254:44739 requested a full tablet report, sending...
I20260809 19:25:34.014624 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 23916
I20260809 19:25:34.014665 23695 ts_manager.cc:194] Registered new tserver with Master: 61d635e1443c497dba9326ab5b7de3a6 (127.21.81.194:41575)
I20260809 19:25:34.014739 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:25:34.014923 21831 external_mini_cluster.cc:1428] Setting key 18ef3aa5ddffb60801a6474ea5b047a4
I20260809 19:25:34.015154 23695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:56943
I20260809 19:25:34.020628 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.015265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:36133 (local address 127.21.81.254:44739)
0809 19:25:34.015303 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.015305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.015312 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:25:34.015329 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.015329 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.015338 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.015400 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:34.015823 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.016349 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.017004 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.017206 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.017287 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.018204 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:34.018205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:34.018206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:34.018212 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:34.019472 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:34.019853 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:34.019854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:34.019854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:34.019911 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:34.020101 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:34.020102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:34.020102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:34.020245 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:34.020314 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.020470 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.020544 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:25:34.020649 24060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.015036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:44739 (local address 127.21.81.194:36133)
0809 19:25:34.015152 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.015163 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.015210 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.015442 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.015443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.015660 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:34.015719 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.015721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.016443 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.016444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.016924 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.016925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.017339 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.017339 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.017393 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.017844 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:34.017849 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:34.018097 (+ 248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:34.019584 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:34.019585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:34.019585 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:34.019771 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:34.019977 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:34.019978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:34.019978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:34.020038 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:34.020319 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:34.020320 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:34.020364 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.020445 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.020545 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:25:34.041149 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:44739
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:39449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:34.123410 24063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:34.123617 24063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:34.123690 24063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:34.124940 24063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:34.125018 24063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:34.125041 24063 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 19:25:34.125056 24063 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:25:34.125074 24063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:25:34.126684 24063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:39449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.81.254:44739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:34.126899 24063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:34.127108 24063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:34.132225 24069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:34.132272 24068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:34.132283 24063 server_base.cc:1034] running on GCE node
W20260809 19:25:34.132412 24072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:34.132697 24063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:34.132953 24063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:34.134083 24063 hybrid_clock.cc:648] HybridClock initialized: now 1786303534134071 us; error 26 us; skew 500 ppm
Aug 09 19:25:34 dist-test-slave-njxd krb5kdc[23648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303534, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:34.137298 24063 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:25:34.137887 24063 webserver.cc:460] Webserver started at http://127.21.81.195:42499/ using document root <none> and password file <none>
I20260809 19:25:34.138202 24063 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:34.138279 24063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:34.138450 24063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:34.139748 24063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "dd900b01a0a9452ea03991f44761a8bf"
format_stamp: "Formatted at 2026-08-09 19:25:34 on dist-test-slave-njxd"
server_key: "a5d9722097738f0958a70aec8c8c52a0"
server_key_iv: "8197432eec0160a2293bc052b9db63e1"
server_key_version: "encryptionkey@0"
I20260809 19:25:34.140110 24063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "dd900b01a0a9452ea03991f44761a8bf"
format_stamp: "Formatted at 2026-08-09 19:25:34 on dist-test-slave-njxd"
server_key: "a5d9722097738f0958a70aec8c8c52a0"
server_key_iv: "8197432eec0160a2293bc052b9db63e1"
server_key_version: "encryptionkey@0"
I20260809 19:25:34.141446 24063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:25:34.142227 24079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:34.142421 24063 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:34.142509 24063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "dd900b01a0a9452ea03991f44761a8bf"
format_stamp: "Formatted at 2026-08-09 19:25:34 on dist-test-slave-njxd"
server_key: "a5d9722097738f0958a70aec8c8c52a0"
server_key_iv: "8197432eec0160a2293bc052b9db63e1"
server_key_version: "encryptionkey@0"
I20260809 19:25:34.142565 24063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:34.163697 24063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:34.165509 24063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:34.165738 24063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:34.166321 24063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:34.166363 24063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:34.166384 24063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:34.166419 24063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:25:34 dist-test-slave-njxd krb5kdc[23648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303534, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:34.173991 24063 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:39711
I20260809 19:25:34.174091 24198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:39711 every 8 connection(s)
I20260809 19:25:34.174378 24063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:25:34.176306 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 24063
I20260809 19:25:34.176400 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:25:34.176643 21831 external_mini_cluster.cc:1428] Setting key 8ff3580abd59a523728d20c6a6a6788a
I20260809 19:25:34.177418 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.166846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:51971 (local address 127.21.81.254:44739)
0809 19:25:34.166901 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.166903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.167158 (+ 255us) server_negotiation.cc:407] Connection header received
0809 19:25:34.167675 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.167677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.167690 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.167752 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:34.168685 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.169474 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.171177 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.171485 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.171562 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.174527 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:34.174529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:34.174529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:34.174537 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:34.175827 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:34.176204 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:34.176205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:34.176205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:34.176279 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:34.176567 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:34.176569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:34.176569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:34.176723 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:34.176800 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.177227 (+ 427us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.177315 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:25:34.177587 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.166843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:44739 (local address 127.21.81.195:51971)
0809 19:25:34.167038 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.167074 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.167564 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.167875 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.167880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.168274 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:34.168572 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.168579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.169596 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.169598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.171084 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.171087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.171585 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.171586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.171744 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.172297 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:34.172310 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:34.174400 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:34.175899 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:34.175905 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:34.175907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:34.176120 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:34.176353 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:34.176353 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:34.176353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:34.176506 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:34.176874 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:34.176880 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:34.177060 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.177186 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.177347 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":124,"threads_started":1}
I20260809 19:25:34.178073 24199 heartbeater.cc:347] Connected to a master server at 127.21.81.254:44739
I20260809 19:25:34.178174 24199 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:34.178376 24199 heartbeater.cc:511] Master 127.21.81.254:44739 requested a full tablet report, sending...
I20260809 19:25:34.178781 23695 ts_manager.cc:194] Registered new tserver with Master: dd900b01a0a9452ea03991f44761a8bf (127.21.81.195:39711)
I20260809 19:25:34.179289 23695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:51971
I20260809 19:25:34.184000 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.179119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:42831 (local address 127.21.81.254:44739)
0809 19:25:34.179163 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.179165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.179221 (+ 56us) server_negotiation.cc:407] Connection header received
0809 19:25:34.179290 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.179290 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.179298 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.179341 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:34.179733 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.180178 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.180914 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.181088 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.181146 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.181983 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:34.181984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:34.181984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:34.181990 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:34.183142 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:34.183434 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:34.183434 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:34.183435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:34.183481 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:34.183644 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:34.183644 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:34.183645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:34.183742 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:34.183790 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.183889 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.183942 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:25:34.184015 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.179023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:44739 (local address 127.21.81.195:42831)
0809 19:25:34.179166 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.179177 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.179237 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.179404 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.179405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.179579 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:34.179655 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.179657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.180282 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.180282 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.180850 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.180853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.181162 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.181163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.181221 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.181562 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:34.181566 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:34.181911 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:34.183210 (+ 1299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:34.183211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:34.183211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:34.183371 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:34.183537 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:34.183538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:34.183538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:34.183592 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:34.183793 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:34.183794 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:34.183828 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.183876 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.183949 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260809 19:25:34.188386 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 19:25:34 dist-test-slave-njxd krb5kdc[23648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 19:25:34 dist-test-slave-njxd krb5kdc[23648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:34.211074 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.204401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50046 (local address 127.21.81.254:44739)
0809 19:25:34.204464 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.204466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.204482 (+ 16us) server_negotiation.cc:407] Connection header received
0809 19:25:34.204521 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.204522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.204531 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.204583 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:34.205120 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.205698 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.206391 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.206567 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.206612 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.208791 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:34.208793 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:34.208793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:34.208799 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:34.210209 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:34.210491 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:34.210491 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:34.210492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:34.210542 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:34.210689 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:34.210690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:34.210690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:34.210779 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:34.210850 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.210926 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.210982 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:25:34.212641 23695 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:25:34.214471 23695 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:25:34.221223 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.216940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:41263 (local address 127.0.0.1:53896)
0809 19:25:34.217344 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.217353 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.217390 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.217790 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.217792 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.217794 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:34.217852 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.217854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.218800 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.218801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.220075 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.220078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.220862 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.220863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.220959 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.220974 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.221068 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.221128 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":1,"thread_start_us":97,"threads_started":1}
I20260809 19:25:34.221227 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.217076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53896 (local address 127.21.81.193:41263)
0809 19:25:34.217484 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.217488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.217499 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:25:34.217538 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.217539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.217608 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.217705 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:34.217938 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.218703 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.220159 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.220756 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.220860 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.220927 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.221086 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.221136 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":102,"threads_started":1}
I20260809 19:25:34.221612 24222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.217254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60472 (local address 127.21.81.194:41575)
0809 19:25:34.217433 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.217438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.217787 (+ 349us) server_negotiation.cc:407] Connection header received
0809 19:25:34.217835 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.217837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.217852 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.217919 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:34.218121 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.219163 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.220342 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.221056 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.221120 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.221177 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.221438 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.221483 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
I20260809 19:25:34.221393 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.216894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:39711 (local address 127.0.0.1:52438)
0809 19:25:34.217104 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.217151 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.217215 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.217713 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.217717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.217719 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:34.217789 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.217792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.219115 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.219116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.220165 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.220167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.221139 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.221140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.221220 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.221224 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.221256 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.221307 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":68,"threads_started":1}
I20260809 19:25:34.221400 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.217106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:41575 (local address 127.0.0.1:60472)
0809 19:25:34.217660 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.217671 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.217722 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.217988 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.217989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.217991 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:34.218027 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.218028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.219241 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.219242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.220253 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.220256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.221206 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.221206 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.221264 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.221267 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.221311 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.221340 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":81,"threads_started":1}
I20260809 19:25:34.221792 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.217276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52438 (local address 127.21.81.195:39711)
0809 19:25:34.217493 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.217504 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.217512 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:34.217543 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.217545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.217576 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.217874 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:34.217942 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.219028 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.220253 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.221025 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.221478 (+ 453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.221550 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.221627 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.221676 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":90,"threads_started":1}
I20260809 19:25:34.222489 23838 tablet_service.cc:1467] Processing CreateTablet for tablet 96e2c25da9544df0ac6426d03332d1b0 (DEFAULT_TABLE table=test-table [id=febc4fe2e6504a0b9a6567f9a2662656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:34.222492 24130 tablet_service.cc:1467] Processing CreateTablet for tablet 96e2c25da9544df0ac6426d03332d1b0 (DEFAULT_TABLE table=test-table [id=febc4fe2e6504a0b9a6567f9a2662656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:34.222900 24130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e2c25da9544df0ac6426d03332d1b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:34.222900 23838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e2c25da9544df0ac6426d03332d1b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:34.226091 23985 tablet_service.cc:1467] Processing CreateTablet for tablet 96e2c25da9544df0ac6426d03332d1b0 (DEFAULT_TABLE table=test-table [id=febc4fe2e6504a0b9a6567f9a2662656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:34.226462 23985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e2c25da9544df0ac6426d03332d1b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:34.226845 24225 tablet_bootstrap.cc:492] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05: Bootstrap starting.
I20260809 19:25:34.227453 24225 tablet_bootstrap.cc:654] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:34.227768 24225 log.cc:824] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:34.228503 24225 tablet_bootstrap.cc:492] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05: No bootstrap required, opened a new log
I20260809 19:25:34.228560 24225 ts_tablet_manager.cc:1397] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:25:34.229468 24227 tablet_bootstrap.cc:492] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6: Bootstrap starting.
I20260809 19:25:34.229647 24224 tablet_bootstrap.cc:492] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf: Bootstrap starting.
I20260809 19:25:34.230170 24227 tablet_bootstrap.cc:654] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:34.230530 24227 log.cc:824] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:34.230557 24224 tablet_bootstrap.cc:654] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:34.230474 24225 raft_consensus.cc:348] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } }
I20260809 19:25:34.230638 24225 raft_consensus.cc:374] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:34.230685 24225 raft_consensus.cc:729] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecbb40c4b84f4804bfd5d54d340dfa05, State: Initialized, Role: FOLLOWER
I20260809 19:25:34.230818 24225 consensus_queue.cc:260] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } }
I20260809 19:25:34.230909 24224 log.cc:824] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:34.231130 24225 ts_tablet_manager.cc:1428] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:25:34.231268 23906 heartbeater.cc:503] Master 127.21.81.254:44739 was elected leader, sending a full tablet report...
I20260809 19:25:34.231341 24227 tablet_bootstrap.cc:492] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6: No bootstrap required, opened a new log
I20260809 19:25:34.231412 24227 ts_tablet_manager.cc:1397] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:25:34.231771 24224 tablet_bootstrap.cc:492] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf: No bootstrap required, opened a new log
I20260809 19:25:34.231848 24224 ts_tablet_manager.cc:1397] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:25:34.233433 24227 raft_consensus.cc:348] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } }
I20260809 19:25:34.233620 24227 raft_consensus.cc:374] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:34.233680 24227 raft_consensus.cc:729] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d635e1443c497dba9326ab5b7de3a6, State: Initialized, Role: FOLLOWER
I20260809 19:25:34.233796 24227 consensus_queue.cc:260] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } }
I20260809 19:25:34.233914 24224 raft_consensus.cc:348] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } }
I20260809 19:25:34.234076 24227 ts_tablet_manager.cc:1428] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 19:25:34.234074 24224 raft_consensus.cc:374] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:34.234128 24224 raft_consensus.cc:729] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd900b01a0a9452ea03991f44761a8bf, State: Initialized, Role: FOLLOWER
I20260809 19:25:34.234138 24054 heartbeater.cc:503] Master 127.21.81.254:44739 was elected leader, sending a full tablet report...
I20260809 19:25:34.234285 24224 consensus_queue.cc:260] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } }
I20260809 19:25:34.234588 24224 ts_tablet_manager.cc:1428] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 19:25:34.234615 24199 heartbeater.cc:503] Master 127.21.81.254:44739 was elected leader, sending a full tablet report...
W20260809 19:25:34.261226 24055 tablet.cc:2369] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:25:34.351497 23908 tablet.cc:2369] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:25:34.396548 24232 raft_consensus.cc:482] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:25:34.396694 24232 raft_consensus.cc:504] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } }
I20260809 19:25:34.397182 24232 leader_election.cc:288] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d635e1443c497dba9326ab5b7de3a6 (127.21.81.194:41575), ecbb40c4b84f4804bfd5d54d340dfa05 (127.21.81.193:41263)
I20260809 19:25:34.400111 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.397396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:41263 (local address 127.21.81.195:34483)
0809 19:25:34.397461 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.397476 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.397529 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.397726 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.397727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.397738 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:34.397818 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.397820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.398499 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.398500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.399274 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.399276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.399819 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.399820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.399903 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.399906 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.399962 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.399990 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":11}
I20260809 19:25:34.400166 24222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.397489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:36021 (local address 127.21.81.194:41575)
0809 19:25:34.397549 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.397551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.397709 (+ 158us) server_negotiation.cc:407] Connection header received
0809 19:25:34.397746 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.397747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.397756 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.397817 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:34.397995 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.398469 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.399409 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.399789 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.399862 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.399869 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.400045 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.400091 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:25:34.400230 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.397406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:41575 (local address 127.21.81.195:36021)
0809 19:25:34.397613 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.397623 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.397698 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.397830 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.397831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.397832 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:34.397923 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.397924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.398558 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.398559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.399285 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.399287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.399879 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.399880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.399959 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.399964 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.400023 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.400060 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":1,"thread_start_us":122,"threads_started":1}
I20260809 19:25:34.400104 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.397482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:34483 (local address 127.21.81.193:41263)
0809 19:25:34.397535 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.397537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.397554 (+ 17us) server_negotiation.cc:407] Connection header received
0809 19:25:34.397615 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.397615 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.397630 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.397697 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:34.397901 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.398410 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.399372 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.399735 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.399790 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.399797 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.399981 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.400030 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:25:34.400637 24006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e2c25da9544df0ac6426d03332d1b0" candidate_uuid: "dd900b01a0a9452ea03991f44761a8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d635e1443c497dba9326ab5b7de3a6" is_pre_election: true
I20260809 19:25:34.400735 23858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e2c25da9544df0ac6426d03332d1b0" candidate_uuid: "dd900b01a0a9452ea03991f44761a8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" is_pre_election: true
I20260809 19:25:34.400826 24006 raft_consensus.cc:2454] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd900b01a0a9452ea03991f44761a8bf in term 0.
I20260809 19:25:34.400857 23858 raft_consensus.cc:2454] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd900b01a0a9452ea03991f44761a8bf in term 0.
I20260809 19:25:34.401096 24081 leader_election.cc:302] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd900b01a0a9452ea03991f44761a8bf, ecbb40c4b84f4804bfd5d54d340dfa05; no voters:
I20260809 19:25:34.401268 24232 raft_consensus.cc:2781] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:25:34.401328 24232 raft_consensus.cc:482] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:25:34.401351 24232 raft_consensus.cc:3037] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:34.402190 24232 raft_consensus.cc:504] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } }
I20260809 19:25:34.402339 24232 leader_election.cc:288] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [CANDIDATE]: Term 1 election: Requested vote from peers 61d635e1443c497dba9326ab5b7de3a6 (127.21.81.194:41575), ecbb40c4b84f4804bfd5d54d340dfa05 (127.21.81.193:41263)
I20260809 19:25:34.402499 23858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e2c25da9544df0ac6426d03332d1b0" candidate_uuid: "dd900b01a0a9452ea03991f44761a8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05"
I20260809 19:25:34.402509 24006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e2c25da9544df0ac6426d03332d1b0" candidate_uuid: "dd900b01a0a9452ea03991f44761a8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d635e1443c497dba9326ab5b7de3a6"
I20260809 19:25:34.402590 23858 raft_consensus.cc:3037] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:34.402619 24006 raft_consensus.cc:3037] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:34.403395 23858 raft_consensus.cc:2454] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd900b01a0a9452ea03991f44761a8bf in term 1.
I20260809 19:25:34.403455 24006 raft_consensus.cc:2454] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd900b01a0a9452ea03991f44761a8bf in term 1.
I20260809 19:25:34.403589 24081 leader_election.cc:302] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd900b01a0a9452ea03991f44761a8bf, ecbb40c4b84f4804bfd5d54d340dfa05; no voters:
I20260809 19:25:34.403678 24232 raft_consensus.cc:2781] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:34.403828 24232 raft_consensus.cc:686] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [term 1 LEADER]: Becoming Leader. State: Replica: dd900b01a0a9452ea03991f44761a8bf, State: Running, Role: LEADER
I20260809 19:25:34.403956 24232 consensus_queue.cc:237] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } }
I20260809 19:25:34.404882 23694 catalog_manager.cc:5495] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf reported cstate change: term changed from 0 to 1, leader changed from <none> to dd900b01a0a9452ea03991f44761a8bf (127.21.81.195). New cstate: current_term: 1 leader_uuid: "dd900b01a0a9452ea03991f44761a8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } health_report { overall_health: UNKNOWN } } }
W20260809 19:25:34.425493 24201 tablet.cc:2369] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 19:25:34 dist-test-slave-njxd krb5kdc[23648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303533, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:34.473492 24241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.466485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:44739 (local address 127.0.0.1:50056)
0809 19:25:34.466660 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.466670 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.466747 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.466889 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.466890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.467210 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:34.467273 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.467275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.467906 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.467906 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.468428 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.468429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.468734 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.468735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.468776 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.469269 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:34.469274 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:34.470718 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:34.472517 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:34.472525 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:34.472527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:34.472727 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:34.472907 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:34.472908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:34.472908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:34.472964 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:34.473166 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:34.473172 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:34.473211 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.473270 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.473373 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
I20260809 19:25:34.473496 23758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.466572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50056 (local address 127.21.81.254:44739)
0809 19:25:34.466676 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.466681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.466736 (+ 55us) server_negotiation.cc:407] Connection header received
0809 19:25:34.466801 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.466802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.466812 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.466866 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:34.467344 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.467834 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.468497 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.468670 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.468721 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.470799 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:34.470801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:34.470801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:34.470807 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:34.472441 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:34.472796 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:34.472797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:34.472797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:34.472848 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:34.473016 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:34.473016 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:34.473016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:34.473114 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:34.473163 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.473313 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.473420 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":73}
I20260809 19:25:34.475788 23694 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:50056:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 19:25:34.476032 23694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:25:34.477618 23985 tablet_service.cc:1467] Processing CreateTablet for tablet 5c7ca2af5cae444d812a2bea36a92702 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3c6a3f788f9b4e1d9027efd7340cea95]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:25:34.477649 23838 tablet_service.cc:1467] Processing CreateTablet for tablet 5c7ca2af5cae444d812a2bea36a92702 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3c6a3f788f9b4e1d9027efd7340cea95]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:25:34.477764 24130 tablet_service.cc:1467] Processing CreateTablet for tablet 5c7ca2af5cae444d812a2bea36a92702 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3c6a3f788f9b4e1d9027efd7340cea95]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:25:34.477814 23985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c7ca2af5cae444d812a2bea36a92702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:34.477814 23838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c7ca2af5cae444d812a2bea36a92702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:34.477895 24130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c7ca2af5cae444d812a2bea36a92702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:34.479136 24224 tablet_bootstrap.cc:492] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf: Bootstrap starting.
I20260809 19:25:34.479393 24225 tablet_bootstrap.cc:492] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05: Bootstrap starting.
I20260809 19:25:34.479619 24224 tablet_bootstrap.cc:654] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:34.479800 24225 tablet_bootstrap.cc:654] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:34.480216 24227 tablet_bootstrap.cc:492] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6: Bootstrap starting.
I20260809 19:25:34.480405 24224 tablet_bootstrap.cc:492] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf: No bootstrap required, opened a new log
I20260809 19:25:34.480465 24224 ts_tablet_manager.cc:1397] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 19:25:34.480650 24224 raft_consensus.cc:348] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.480738 24224 raft_consensus.cc:374] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:34.480762 24224 raft_consensus.cc:729] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd900b01a0a9452ea03991f44761a8bf, State: Initialized, Role: FOLLOWER
I20260809 19:25:34.480811 24224 consensus_queue.cc:260] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.480921 24224 ts_tablet_manager.cc:1428] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:34.480971 24227 tablet_bootstrap.cc:654] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:34.480958 24232 tablet_replica.cc:439] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } }
I20260809 19:25:34.481066 24232 tablet_replica.cc:442] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:25:34.481556 24225 tablet_bootstrap.cc:492] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05: No bootstrap required, opened a new log
I20260809 19:25:34.481680 24225 ts_tablet_manager.cc:1397] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:25:34.481729 24227 tablet_bootstrap.cc:492] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6: No bootstrap required, opened a new log
I20260809 19:25:34.481775 24227 ts_tablet_manager.cc:1397] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 19:25:34.481850 24225 raft_consensus.cc:348] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.481901 24225 raft_consensus.cc:374] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:34.481915 24225 raft_consensus.cc:729] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecbb40c4b84f4804bfd5d54d340dfa05, State: Initialized, Role: FOLLOWER
I20260809 19:25:34.481935 24227 raft_consensus.cc:348] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.481974 24225 consensus_queue.cc:260] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.482017 24227 raft_consensus.cc:374] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:34.482043 24227 raft_consensus.cc:729] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d635e1443c497dba9326ab5b7de3a6, State: Initialized, Role: FOLLOWER
I20260809 19:25:34.482066 24225 ts_tablet_manager.cc:1428] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:34.482087 24227 consensus_queue.cc:260] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.482144 24229 tablet_replica.cc:439] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } }
I20260809 19:25:34.482196 24227 ts_tablet_manager.cc:1428] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:34.482226 24229 tablet_replica.cc:442] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:25:34.482338 24231 tablet_replica.cc:439] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } }
I20260809 19:25:34.482417 24231 tablet_replica.cc:442] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:25:34.548297 24232 raft_consensus.cc:482] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:25:34.548429 24232 raft_consensus.cc:504] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.548583 24232 leader_election.cc:288] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ecbb40c4b84f4804bfd5d54d340dfa05 (127.21.81.193:41263), 61d635e1443c497dba9326ab5b7de3a6 (127.21.81.194:41575)
I20260809 19:25:34.548827 23858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7ca2af5cae444d812a2bea36a92702" candidate_uuid: "dd900b01a0a9452ea03991f44761a8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" is_pre_election: true
I20260809 19:25:34.548851 24006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7ca2af5cae444d812a2bea36a92702" candidate_uuid: "dd900b01a0a9452ea03991f44761a8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d635e1443c497dba9326ab5b7de3a6" is_pre_election: true
I20260809 19:25:34.548933 24006 raft_consensus.cc:2454] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd900b01a0a9452ea03991f44761a8bf in term 0.
I20260809 19:25:34.548942 23858 raft_consensus.cc:2454] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd900b01a0a9452ea03991f44761a8bf in term 0.
I20260809 19:25:34.549153 24083 leader_election.cc:302] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d635e1443c497dba9326ab5b7de3a6, dd900b01a0a9452ea03991f44761a8bf; no voters:
I20260809 19:25:34.549295 24232 raft_consensus.cc:2781] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:25:34.549348 24232 raft_consensus.cc:482] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:25:34.549373 24232 raft_consensus.cc:3037] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:34.550120 24232 raft_consensus.cc:504] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.550277 24232 leader_election.cc:288] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [CANDIDATE]: Term 1 election: Requested vote from peers ecbb40c4b84f4804bfd5d54d340dfa05 (127.21.81.193:41263), 61d635e1443c497dba9326ab5b7de3a6 (127.21.81.194:41575)
I20260809 19:25:34.550449 23858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7ca2af5cae444d812a2bea36a92702" candidate_uuid: "dd900b01a0a9452ea03991f44761a8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05"
I20260809 19:25:34.550447 24006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c7ca2af5cae444d812a2bea36a92702" candidate_uuid: "dd900b01a0a9452ea03991f44761a8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d635e1443c497dba9326ab5b7de3a6"
I20260809 19:25:34.550529 23858 raft_consensus.cc:3037] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:34.550529 24006 raft_consensus.cc:3037] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:34.551290 24006 raft_consensus.cc:2454] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd900b01a0a9452ea03991f44761a8bf in term 1.
I20260809 19:25:34.551290 23858 raft_consensus.cc:2454] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd900b01a0a9452ea03991f44761a8bf in term 1.
I20260809 19:25:34.551460 24081 leader_election.cc:302] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd900b01a0a9452ea03991f44761a8bf, ecbb40c4b84f4804bfd5d54d340dfa05; no voters:
I20260809 19:25:34.551553 24232 raft_consensus.cc:2781] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:34.551625 24232 raft_consensus.cc:686] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [term 1 LEADER]: Becoming Leader. State: Replica: dd900b01a0a9452ea03991f44761a8bf, State: Running, Role: LEADER
I20260809 19:25:34.551678 24232 consensus_queue.cc:237] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.552033 24234 tablet_replica.cc:439] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf: TxnStatusTablet state changed. Reason: New leader dd900b01a0a9452ea03991f44761a8bf. Latest consensus state: current_term: 1 leader_uuid: "dd900b01a0a9452ea03991f44761a8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } }
I20260809 19:25:34.552145 24234 tablet_replica.cc:442] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:25:34.552451 24246 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 19:25:34.552428 23694 catalog_manager.cc:5495] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf reported cstate change: term changed from 0 to 1, leader changed from <none> to dd900b01a0a9452ea03991f44761a8bf (127.21.81.195). New cstate: current_term: 1 leader_uuid: "dd900b01a0a9452ea03991f44761a8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } health_report { overall_health: HEALTHY } } }
I20260809 19:25:34.552588 24246 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 19:25:34.576126 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.573319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52446 (local address 127.21.81.195:39711)
0809 19:25:34.573365 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.573367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.573397 (+ 30us) server_negotiation.cc:407] Connection header received
0809 19:25:34.573482 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.573483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.573498 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.573556 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:34.573730 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.574197 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.575354 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.575559 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.575612 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.575825 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:34.575913 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:34.575970 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.576048 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.576066 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:25:34.576180 24241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.573252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:39711 (local address 127.0.0.1:52446)
0809 19:25:34.573348 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.573360 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.573413 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.573604 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.573605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.573607 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:25:34.573662 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.573665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.574279 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.574280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.575223 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.575226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.575635 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.575636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.575709 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.575728 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:25:34.575978 (+ 250us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:25:34.575987 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.576043 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.576097 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 19:25:34.578086 24006 raft_consensus.cc:1261] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Refusing update from remote peer dd900b01a0a9452ea03991f44761a8bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:34.578123 23858 raft_consensus.cc:1261] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 FOLLOWER]: Refusing update from remote peer dd900b01a0a9452ea03991f44761a8bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:34.578478 24234 consensus_queue.cc:1035] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:34.578630 24232 consensus_queue.cc:1035] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:34.579913 24231 tablet_replica.cc:439] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6: TxnStatusTablet state changed. Reason: New leader dd900b01a0a9452ea03991f44761a8bf. Latest consensus state: current_term: 1 leader_uuid: "dd900b01a0a9452ea03991f44761a8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } }
I20260809 19:25:34.579931 24229 tablet_replica.cc:439] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05: TxnStatusTablet state changed. Reason: New leader dd900b01a0a9452ea03991f44761a8bf. Latest consensus state: current_term: 1 leader_uuid: "dd900b01a0a9452ea03991f44761a8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } }
I20260809 19:25:34.580013 24231 tablet_replica.cc:442] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:25:34.580026 24229 tablet_replica.cc:442] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:25:34.581197 24232 tablet_replica.cc:439] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd900b01a0a9452ea03991f44761a8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } }
I20260809 19:25:34.581285 24232 tablet_replica.cc:442] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:25:34.581370 24229 tablet_replica.cc:439] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd900b01a0a9452ea03991f44761a8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } }
I20260809 19:25:34.581445 24229 tablet_replica.cc:442] T 5c7ca2af5cae444d812a2bea36a92702 P ecbb40c4b84f4804bfd5d54d340dfa05: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:25:34.581483 24250 mvcc.cc:205] Tried to move back new op lower bound from 7316699277629026304 to 7316699277524074496. Current Snapshot: MvccSnapshot[applied={T|T < 7316699277629026304}]
I20260809 19:25:34.585526 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.583274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.21.81.195:39711)
0809 19:25:34.583320 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.583334 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.583343 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:25:34.583394 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.583395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.583406 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.583457 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:34.583602 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.584039 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.584828 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.585025 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.585105 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.585266 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:34.585307 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:34.585374 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.585430 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.585452 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:25:34.590360 24234 tablet_replica.cc:439] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd900b01a0a9452ea03991f44761a8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } }
I20260809 19:25:34.590462 24234 tablet_replica.cc:442] T 5c7ca2af5cae444d812a2bea36a92702 P dd900b01a0a9452ea03991f44761a8bf: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:25:34.590785 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.588112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:39711 (local address 127.21.81.195:38565)
0809 19:25:34.588193 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.588204 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.588249 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.588370 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.588371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.588372 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:25:34.588433 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.588434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.589273 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.589273 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.589851 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.589852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.590285 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.590286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.590351 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.590365 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:25:34.590600 (+ 235us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:25:34.590601 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.590645 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.590690 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 19:25:34.590848 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.588179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:38565 (local address 127.21.81.195:39711)
0809 19:25:34.588213 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.588215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.588245 (+ 30us) server_negotiation.cc:407] Connection header received
0809 19:25:34.588291 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.588292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.588301 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.588365 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:34.588491 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.589182 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.589936 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.590200 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.590283 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.590468 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:34.590499 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:34.590540 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.590709 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.590728 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:25:34.591187 24231 tablet_replica.cc:439] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd900b01a0a9452ea03991f44761a8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } } }
I20260809 19:25:34.591288 24231 tablet_replica.cc:442] T 5c7ca2af5cae444d812a2bea36a92702 P 61d635e1443c497dba9326ab5b7de3a6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:25:34.593158 23858 raft_consensus.cc:1261] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 FOLLOWER]: Refusing update from remote peer dd900b01a0a9452ea03991f44761a8bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:34.593215 24006 raft_consensus.cc:1261] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Refusing update from remote peer dd900b01a0a9452ea03991f44761a8bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:34.593427 24261 consensus_queue.cc:1035] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:34.593523 24232 consensus_queue.cc:1035] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:34.595000 24249 mvcc.cc:205] Tried to move back new op lower bound from 7316699277692047360 to 7316699276919214080. Current Snapshot: MvccSnapshot[applied={T|T < 7316699277692047360}]
I20260809 19:25:34.597064 24251 mvcc.cc:205] Tried to move back new op lower bound from 7316699277692047360 to 7316699276919214080. Current Snapshot: MvccSnapshot[applied={T|T < 7316699277692047360}]
I20260809 19:25:34.605033 23695 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:34.605134 23695 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:34.605870 23695 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862: Sending DeleteTablet for 3 replicas of tablet 96e2c25da9544df0ac6426d03332d1b0
I20260809 19:25:34.606248 23985 tablet_service.cc:1514] Processing DeleteTablet for tablet 96e2c25da9544df0ac6426d03332d1b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:34 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:60472
I20260809 19:25:34.606310 23838 tablet_service.cc:1514] Processing DeleteTablet for tablet 96e2c25da9544df0ac6426d03332d1b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:34 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:53896
I20260809 19:25:34.606472 24130 tablet_service.cc:1514] Processing DeleteTablet for tablet 96e2c25da9544df0ac6426d03332d1b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:34 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:52438
I20260809 19:25:34.606611 24267 tablet_replica.cc:330] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05: stopping tablet replica
I20260809 19:25:34.606721 24268 tablet_replica.cc:330] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf: stopping tablet replica
I20260809 19:25:34.606803 24267 raft_consensus.cc:2229] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:34.606845 24268 raft_consensus.cc:2229] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:25:34.606792 23695 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 19:25:34.606932 24268 raft_consensus.cc:2258] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 19:25:34.606971 23695 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:25:34.607249 24267 raft_consensus.cc:2258] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:34.607395 24268 ts_tablet_manager.cc:1905] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:34.607813 24267 ts_tablet_manager.cc:1905] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:34.608273 23838 tablet_service.cc:1467] Processing CreateTablet for tablet fb831d7706024a86b0d7c12fe743cb94 (DEFAULT_TABLE table=test-table [id=d63aa090743549379522ed95fdbc59c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:34.608400 23838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb831d7706024a86b0d7c12fe743cb94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:34.609223 24268 ts_tablet_manager.cc:1918] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:25:34.609303 24268 log.cc:1196] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/96e2c25da9544df0ac6426d03332d1b0
I20260809 19:25:34.609565 24268 ts_tablet_manager.cc:1939] T 96e2c25da9544df0ac6426d03332d1b0 P dd900b01a0a9452ea03991f44761a8bf: Deleting consensus metadata
I20260809 19:25:34.609882 24225 tablet_bootstrap.cc:492] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05: Bootstrap starting.
I20260809 19:25:34.610294 24267 ts_tablet_manager.cc:1918] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:25:34.610313 24225 tablet_bootstrap.cc:654] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:34.610543 23985 tablet_service.cc:1467] Processing CreateTablet for tablet fb831d7706024a86b0d7c12fe743cb94 (DEFAULT_TABLE table=test-table [id=d63aa090743549379522ed95fdbc59c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:34.610661 23985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb831d7706024a86b0d7c12fe743cb94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:34.610841 24267 log.cc:1196] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/wal/wals/96e2c25da9544df0ac6426d03332d1b0
I20260809 19:25:34.611061 24225 tablet_bootstrap.cc:492] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05: No bootstrap required, opened a new log
I20260809 19:25:34.611114 24225 ts_tablet_manager.cc:1397] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:25:34.611271 24267 ts_tablet_manager.cc:1939] T 96e2c25da9544df0ac6426d03332d1b0 P ecbb40c4b84f4804bfd5d54d340dfa05: Deleting consensus metadata
I20260809 19:25:34.611265 24225 raft_consensus.cc:348] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.611413 24225 raft_consensus.cc:374] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:34.611442 24225 raft_consensus.cc:729] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecbb40c4b84f4804bfd5d54d340dfa05, State: Initialized, Role: FOLLOWER
I20260809 19:25:34.611482 24225 consensus_queue.cc:260] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.611569 24225 ts_tablet_manager.cc:1428] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:34.611953 23679 catalog_manager.cc:4841] TS ecbb40c4b84f4804bfd5d54d340dfa05 (127.21.81.193:41263): tablet 96e2c25da9544df0ac6426d03332d1b0 (table test-table [id=febc4fe2e6504a0b9a6567f9a2662656]) successfully deleted
I20260809 19:25:34.612156 24227 tablet_bootstrap.cc:492] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6: Bootstrap starting.
I20260809 19:25:34.612524 24227 tablet_bootstrap.cc:654] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:34.613150 23681 catalog_manager.cc:4841] TS dd900b01a0a9452ea03991f44761a8bf (127.21.81.195:39711): tablet 96e2c25da9544df0ac6426d03332d1b0 (table test-table [id=febc4fe2e6504a0b9a6567f9a2662656]) successfully deleted
I20260809 19:25:34.613224 24227 tablet_bootstrap.cc:492] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6: No bootstrap required, opened a new log
I20260809 19:25:34.613270 24227 ts_tablet_manager.cc:1397] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:25:34.613300 24130 tablet_service.cc:1467] Processing CreateTablet for tablet fb831d7706024a86b0d7c12fe743cb94 (DEFAULT_TABLE table=test-table [id=d63aa090743549379522ed95fdbc59c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:34.613404 24130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb831d7706024a86b0d7c12fe743cb94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:34.613410 24227 raft_consensus.cc:348] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.613483 24227 raft_consensus.cc:374] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:34.613509 24227 raft_consensus.cc:729] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d635e1443c497dba9326ab5b7de3a6, State: Initialized, Role: FOLLOWER
I20260809 19:25:34.613548 24227 consensus_queue.cc:260] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.613659 24227 ts_tablet_manager.cc:1428] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:34.613775 24266 tablet_replica.cc:330] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6: stopping tablet replica
I20260809 19:25:34.613952 24266 raft_consensus.cc:2229] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:34.614085 24266 raft_consensus.cc:2258] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:34.614732 24224 tablet_bootstrap.cc:492] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf: Bootstrap starting.
I20260809 19:25:34.615127 24266 ts_tablet_manager.cc:1905] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:34.615165 24224 tablet_bootstrap.cc:654] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:34.615810 24224 tablet_bootstrap.cc:492] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf: No bootstrap required, opened a new log
I20260809 19:25:34.615865 24224 ts_tablet_manager.cc:1397] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 19:25:34.615993 24224 raft_consensus.cc:348] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.616061 24224 raft_consensus.cc:374] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:34.616086 24224 raft_consensus.cc:729] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd900b01a0a9452ea03991f44761a8bf, State: Initialized, Role: FOLLOWER
I20260809 19:25:34.616125 24224 consensus_queue.cc:260] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.616214 24224 ts_tablet_manager.cc:1428] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:34.616869 24266 ts_tablet_manager.cc:1918] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:25:34.616969 24266 log.cc:1196] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/96e2c25da9544df0ac6426d03332d1b0
I20260809 19:25:34.617249 24266 ts_tablet_manager.cc:1939] T 96e2c25da9544df0ac6426d03332d1b0 P 61d635e1443c497dba9326ab5b7de3a6: Deleting consensus metadata
I20260809 19:25:34.617664 23681 catalog_manager.cc:4841] TS 61d635e1443c497dba9326ab5b7de3a6 (127.21.81.194:41575): tablet 96e2c25da9544df0ac6426d03332d1b0 (table test-table [id=febc4fe2e6504a0b9a6567f9a2662656]) successfully deleted
I20260809 19:25:34.854485 24231 raft_consensus.cc:482] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:25:34.854605 24231 raft_consensus.cc:504] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.854974 24231 leader_election.cc:288] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ecbb40c4b84f4804bfd5d54d340dfa05 (127.21.81.193:41263), dd900b01a0a9452ea03991f44761a8bf (127.21.81.195:39711)
I20260809 19:25:34.858327 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.855300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:37861 (local address 127.21.81.195:39711)
0809 19:25:34.855380 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.855382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.855393 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:25:34.855496 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.855497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.855507 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.855541 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:34.855799 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.856263 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.857353 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.857873 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.857950 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.857962 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.858197 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.858245 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 19:25:34.858333 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.855218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:40627 (local address 127.21.81.193:41263)
0809 19:25:34.855273 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.855276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.855409 (+ 133us) server_negotiation.cc:407] Connection header received
0809 19:25:34.855501 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.855502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.855511 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.855586 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:34.855784 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.856263 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.857358 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.857879 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.857953 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.857962 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.858199 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.858247 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:25:34.858642 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.855152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:41263 (local address 127.21.81.194:40627)
0809 19:25:34.855321 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.855333 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.855422 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.855609 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.855610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.855612 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:34.855704 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.855705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.856356 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.856357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.857254 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.857256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.858000 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.858002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.858097 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.858102 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.858468 (+ 366us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.858515 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":33,"thread_start_us":86,"threads_started":1}
I20260809 19:25:34.858940 23858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb831d7706024a86b0d7c12fe743cb94" candidate_uuid: "61d635e1443c497dba9326ab5b7de3a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" is_pre_election: true
I20260809 19:25:34.859036 23858 raft_consensus.cc:2454] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d635e1443c497dba9326ab5b7de3a6 in term 0.
I20260809 19:25:34.859054 24272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.855193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:39711 (local address 127.21.81.194:37861)
0809 19:25:34.855331 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.855342 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.855404 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.855622 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.855622 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.855625 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:34.855706 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.855707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.856355 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.856355 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.857256 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.857259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.858017 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.858018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.858101 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.858105 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.858921 (+ 816us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.858974 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":75,"thread_start_us":63,"threads_started":1}
I20260809 19:25:34.859313 23935 leader_election.cc:302] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d635e1443c497dba9326ab5b7de3a6, ecbb40c4b84f4804bfd5d54d340dfa05; no voters:
I20260809 19:25:34.859439 24231 raft_consensus.cc:2781] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:25:34.859464 24231 raft_consensus.cc:482] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:25:34.859483 24231 raft_consensus.cc:3037] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:34.860472 24231 raft_consensus.cc:504] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.860616 24231 leader_election.cc:288] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [CANDIDATE]: Term 1 election: Requested vote from peers ecbb40c4b84f4804bfd5d54d340dfa05 (127.21.81.193:41263), dd900b01a0a9452ea03991f44761a8bf (127.21.81.195:39711)
I20260809 19:25:34.860797 23858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb831d7706024a86b0d7c12fe743cb94" candidate_uuid: "61d635e1443c497dba9326ab5b7de3a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05"
I20260809 19:25:34.860893 23858 raft_consensus.cc:3037] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:34.861469 23858 raft_consensus.cc:2454] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d635e1443c497dba9326ab5b7de3a6 in term 1.
I20260809 19:25:34.861658 23935 leader_election.cc:302] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d635e1443c497dba9326ab5b7de3a6, ecbb40c4b84f4804bfd5d54d340dfa05; no voters:
I20260809 19:25:34.861791 24231 raft_consensus.cc:2781] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:34.861948 24231 raft_consensus.cc:686] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 LEADER]: Becoming Leader. State: Replica: 61d635e1443c497dba9326ab5b7de3a6, State: Running, Role: LEADER
I20260809 19:25:34.862027 24231 consensus_queue.cc:237] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:34.860088 24150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb831d7706024a86b0d7c12fe743cb94" candidate_uuid: "61d635e1443c497dba9326ab5b7de3a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd900b01a0a9452ea03991f44761a8bf" is_pre_election: true
I20260809 19:25:34.863147 24150 raft_consensus.cc:2454] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61d635e1443c497dba9326ab5b7de3a6 in term 0.
I20260809 19:25:34.863248 23695 catalog_manager.cc:5495] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 61d635e1443c497dba9326ab5b7de3a6 (127.21.81.194). New cstate: current_term: 1 leader_uuid: "61d635e1443c497dba9326ab5b7de3a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } health_report { overall_health: UNKNOWN } } }
I20260809 19:25:34.865805 24150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb831d7706024a86b0d7c12fe743cb94" candidate_uuid: "61d635e1443c497dba9326ab5b7de3a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd900b01a0a9452ea03991f44761a8bf"
I20260809 19:25:34.865888 24150 raft_consensus.cc:3037] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:34.866931 24150 raft_consensus.cc:2454] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61d635e1443c497dba9326ab5b7de3a6 in term 1.
I20260809 19:25:34.902812 24222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.899886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60474 (local address 127.21.81.194:41575)
0809 19:25:34.899937 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.899939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.899977 (+ 38us) server_negotiation.cc:407] Connection header received
0809 19:25:34.900011 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.900011 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.900031 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.900085 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:34.900217 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.901072 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.901854 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.902216 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.902296 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.902401 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:34.902560 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:34.902648 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.902667 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.902692 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:25:34.908943 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.905644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:55615 (local address 127.21.81.195:39711)
0809 19:25:34.905693 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.905695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.905784 (+ 89us) server_negotiation.cc:407] Connection header received
0809 19:25:34.905833 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.905834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.905846 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.905929 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:34.906081 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.906827 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.908132 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.908427 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.908496 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.908739 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:34.908763 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:34.908813 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.908877 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.908892 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:25:34.908999 24278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.905532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:39711 (local address 127.21.81.194:55615)
0809 19:25:34.905722 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.905733 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.905789 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.905940 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.905941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.905942 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:25:34.906012 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.906014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.906904 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.906905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.908035 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.908037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.908566 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.908567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.908638 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.908653 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:25:34.908828 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:25:34.908830 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.908875 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.908911 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":122,"threads_started":1}
I20260809 19:25:34.911329 23858 raft_consensus.cc:1261] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 FOLLOWER]: Refusing update from remote peer 61d635e1443c497dba9326ab5b7de3a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:34.911345 24150 raft_consensus.cc:1261] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Refusing update from remote peer 61d635e1443c497dba9326ab5b7de3a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:34.911633 24231 consensus_queue.cc:1035] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:34.911852 24274 consensus_queue.cc:1035] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:34.919083 24222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.916744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:50739 (local address 127.21.81.194:41575)
0809 19:25:34.916824 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:34.916826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:34.916835 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:25:34.916856 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:34.916857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:34.916868 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:34.916927 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:34.917062 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.917519 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.918310 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.918584 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.918664 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.918867 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:34.918898 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:34.918943 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:34.918997 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 19:25:34.919016 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:25:34.919101 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:34.916529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:41575 (local address 127.21.81.195:50739)
0809 19:25:34.916571 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:34.916584 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:34.916625 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:34.916942 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:34.916943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:34.916944 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:25:34.916991 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.916992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.917668 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.917669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.918224 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:34.918225 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:34.918706 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:34.918707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:34.918768 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:34.918772 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:25:34.918943 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:25:34.918944 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:25:34.918993 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:25:34.919036 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
W20260809 19:25:34.919927 24264 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 19:25:36.444000 23694 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:36.444131 23694 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:36.445039 23694 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862: Sending DeleteTablet for 3 replicas of tablet fb831d7706024a86b0d7c12fe743cb94
I20260809 19:25:36.445326 23838 tablet_service.cc:1514] Processing DeleteTablet for tablet fb831d7706024a86b0d7c12fe743cb94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:36 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:53896
I20260809 19:25:36.445343 23985 tablet_service.cc:1514] Processing DeleteTablet for tablet fb831d7706024a86b0d7c12fe743cb94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:36 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:60472
I20260809 19:25:36.445369 24129 tablet_service.cc:1514] Processing DeleteTablet for tablet fb831d7706024a86b0d7c12fe743cb94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:36 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:52438
I20260809 19:25:36.445596 24309 tablet_replica.cc:330] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf: stopping tablet replica
I20260809 19:25:36.445698 24309 raft_consensus.cc:2229] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:36.445751 24309 raft_consensus.cc:2258] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:36.446023 24309 ts_tablet_manager.cc:1905] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:36.448083 24309 ts_tablet_manager.cc:1918] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 19:25:36.448158 24309 log.cc:1196] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/fb831d7706024a86b0d7c12fe743cb94
I20260809 19:25:36.448391 24309 ts_tablet_manager.cc:1939] T fb831d7706024a86b0d7c12fe743cb94 P dd900b01a0a9452ea03991f44761a8bf: Deleting consensus metadata
I20260809 19:25:36.448944 23681 catalog_manager.cc:4841] TS dd900b01a0a9452ea03991f44761a8bf (127.21.81.195:39711): tablet fb831d7706024a86b0d7c12fe743cb94 (table test-table [id=d63aa090743549379522ed95fdbc59c1]) successfully deleted
I20260809 19:25:36.449314 24308 tablet_replica.cc:330] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05: stopping tablet replica
I20260809 19:25:36.449398 24308 raft_consensus.cc:2229] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:36.449445 24308 raft_consensus.cc:2258] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:36.449559 24307 tablet_replica.cc:330] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6: stopping tablet replica
I20260809 19:25:36.449651 24307 raft_consensus.cc:2229] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:25:36.449759 24307 raft_consensus.cc:2258] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:36.449756 24308 ts_tablet_manager.cc:1905] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:36.449966 24307 ts_tablet_manager.cc:1905] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:36.451745 24308 ts_tablet_manager.cc:1918] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 19:25:36.451790 24307 ts_tablet_manager.cc:1918] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 19:25:36.451814 24308 log.cc:1196] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/wal/wals/fb831d7706024a86b0d7c12fe743cb94
I20260809 19:25:36.451893 24307 log.cc:1196] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/fb831d7706024a86b0d7c12fe743cb94
I20260809 19:25:36.452119 24308 ts_tablet_manager.cc:1939] T fb831d7706024a86b0d7c12fe743cb94 P ecbb40c4b84f4804bfd5d54d340dfa05: Deleting consensus metadata
I20260809 19:25:36.452147 24307 ts_tablet_manager.cc:1939] T fb831d7706024a86b0d7c12fe743cb94 P 61d635e1443c497dba9326ab5b7de3a6: Deleting consensus metadata
I20260809 19:25:36.452471 23679 catalog_manager.cc:4841] TS ecbb40c4b84f4804bfd5d54d340dfa05 (127.21.81.193:41263): tablet fb831d7706024a86b0d7c12fe743cb94 (table test-table [id=d63aa090743549379522ed95fdbc59c1]) successfully deleted
I20260809 19:25:36.452587 23681 catalog_manager.cc:4841] TS 61d635e1443c497dba9326ab5b7de3a6 (127.21.81.194:41575): tablet fb831d7706024a86b0d7c12fe743cb94 (table test-table [id=d63aa090743549379522ed95fdbc59c1]) successfully deleted
Aug 09 19:25:36 dist-test-slave-njxd krb5kdc[23648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303536, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 19:25:36 dist-test-slave-njxd krb5kdc[23648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303536, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:36.473969 24317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:36.463368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50058 (local address 127.21.81.254:44739)
0809 19:25:36.463518 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:36.463520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:36.463581 (+ 61us) server_negotiation.cc:407] Connection header received
0809 19:25:36.463685 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:36.463686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:36.463701 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:36.463774 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:36.464338 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.464858 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.465608 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.465818 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.465894 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:36.471688 (+ 5794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:36.471690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:36.471690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:36.471697 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:36.472985 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:36.473303 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:36.473304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:36.473304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:36.473357 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:36.473517 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:36.473518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:36.473518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:36.473657 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:36.473739 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:36.473836 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 19:25:36.473901 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260809 19:25:36.475608 23694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:25:36.475800 23694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:25:36.477283 23838 tablet_service.cc:1467] Processing CreateTablet for tablet 120007ea7df34b638b49cfdcb4460385 (DEFAULT_TABLE table=test-table [id=58b35875fc8847418bdcbc68d3c63a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:36.477322 24129 tablet_service.cc:1467] Processing CreateTablet for tablet 120007ea7df34b638b49cfdcb4460385 (DEFAULT_TABLE table=test-table [id=58b35875fc8847418bdcbc68d3c63a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:36.477432 23838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 120007ea7df34b638b49cfdcb4460385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:36.477495 24129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 120007ea7df34b638b49cfdcb4460385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:36.477671 23985 tablet_service.cc:1467] Processing CreateTablet for tablet 120007ea7df34b638b49cfdcb4460385 (DEFAULT_TABLE table=test-table [id=58b35875fc8847418bdcbc68d3c63a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:36.477798 23985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 120007ea7df34b638b49cfdcb4460385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:36.478840 24319 tablet_bootstrap.cc:492] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05: Bootstrap starting.
I20260809 19:25:36.479395 24319 tablet_bootstrap.cc:654] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:36.479403 24320 tablet_bootstrap.cc:492] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6: Bootstrap starting.
I20260809 19:25:36.479809 24320 tablet_bootstrap.cc:654] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:36.480062 24319 tablet_bootstrap.cc:492] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05: No bootstrap required, opened a new log
I20260809 19:25:36.480121 24319 ts_tablet_manager.cc:1397] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:36.480132 24321 tablet_bootstrap.cc:492] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf: Bootstrap starting.
I20260809 19:25:36.480283 24319 raft_consensus.cc:348] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.480367 24319 raft_consensus.cc:374] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:36.480404 24319 raft_consensus.cc:729] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecbb40c4b84f4804bfd5d54d340dfa05, State: Initialized, Role: FOLLOWER
I20260809 19:25:36.480479 24319 consensus_queue.cc:260] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.480535 24320 tablet_bootstrap.cc:492] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6: No bootstrap required, opened a new log
I20260809 19:25:36.480576 24320 ts_tablet_manager.cc:1397] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:36.480690 24319 ts_tablet_manager.cc:1428] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:36.480758 24320 raft_consensus.cc:348] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.480845 24320 raft_consensus.cc:374] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:36.480866 24320 raft_consensus.cc:729] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d635e1443c497dba9326ab5b7de3a6, State: Initialized, Role: FOLLOWER
I20260809 19:25:36.480921 24320 consensus_queue.cc:260] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.481038 24320 ts_tablet_manager.cc:1428] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:36.481389 24321 tablet_bootstrap.cc:654] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:36.482167 24321 tablet_bootstrap.cc:492] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf: No bootstrap required, opened a new log
I20260809 19:25:36.482209 24321 ts_tablet_manager.cc:1397] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:25:36.482299 24321 raft_consensus.cc:348] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.482350 24321 raft_consensus.cc:374] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:36.482364 24321 raft_consensus.cc:729] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd900b01a0a9452ea03991f44761a8bf, State: Initialized, Role: FOLLOWER
I20260809 19:25:36.482424 24321 consensus_queue.cc:260] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.482524 24321 ts_tablet_manager.cc:1428] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:36.554898 24324 raft_consensus.cc:482] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:25:36.555006 24324 raft_consensus.cc:504] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.555397 24324 leader_election.cc:288] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d635e1443c497dba9326ab5b7de3a6 (127.21.81.194:41575), dd900b01a0a9452ea03991f44761a8bf (127.21.81.195:39711)
I20260809 19:25:36.558686 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:36.555638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:41575 (local address 127.21.81.193:53397)
0809 19:25:36.555837 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:36.555849 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:36.555906 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:36.556106 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:36.556107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:36.556111 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:36.556174 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:36.556176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.556941 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.556942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:36.557829 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:36.557831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.558412 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.558413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:36.558481 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:36.558484 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:25:36.558545 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 19:25:36.558590 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":131,"threads_started":1}
I20260809 19:25:36.558679 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:36.555720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:53397 (local address 127.21.81.194:41575)
0809 19:25:36.555908 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:36.555910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:36.555920 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:25:36.556002 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:36.556003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:36.556016 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:36.556099 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:36.556272 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.556845 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.557917 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.558323 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.558413 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:36.558423 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:36.558567 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 19:25:36.558610 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":128,"threads_started":1}
I20260809 19:25:36.558943 24005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120007ea7df34b638b49cfdcb4460385" candidate_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d635e1443c497dba9326ab5b7de3a6" is_pre_election: true
I20260809 19:25:36.559042 24005 raft_consensus.cc:2454] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecbb40c4b84f4804bfd5d54d340dfa05 in term 0.
I20260809 19:25:36.559231 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:36.555885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:39711 (local address 127.21.81.193:60617)
0809 19:25:36.556025 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:36.556037 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:36.556077 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:36.556380 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:36.556381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:36.556384 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:36.556731 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:36.556733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.557396 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.557398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:36.558191 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:36.558194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.558931 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.558932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:36.559018 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:36.559021 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:25:36.559072 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:25:36.559118 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260809 19:25:36.559238 23791 leader_election.cc:302] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d635e1443c497dba9326ab5b7de3a6, ecbb40c4b84f4804bfd5d54d340dfa05; no voters:
I20260809 19:25:36.559465 24324 raft_consensus.cc:2781] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:25:36.559515 24324 raft_consensus.cc:482] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:25:36.559530 24324 raft_consensus.cc:3037] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:36.559532 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:36.556060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:60617 (local address 127.21.81.195:39711)
0809 19:25:36.556250 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:36.556252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:36.556259 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:25:36.556283 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:36.556283 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:36.556293 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:36.556375 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:36.556835 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.557314 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.558290 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.558684 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.559381 (+ 697us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:36.559394 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:36.559414 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 19:25:36.559463 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":126,"threads_started":1}
I20260809 19:25:36.559731 24150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120007ea7df34b638b49cfdcb4460385" candidate_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd900b01a0a9452ea03991f44761a8bf" is_pre_election: true
I20260809 19:25:36.559813 24150 raft_consensus.cc:2454] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecbb40c4b84f4804bfd5d54d340dfa05 in term 0.
I20260809 19:25:36.560374 24324 raft_consensus.cc:504] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.560518 24324 leader_election.cc:288] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [CANDIDATE]: Term 1 election: Requested vote from peers 61d635e1443c497dba9326ab5b7de3a6 (127.21.81.194:41575), dd900b01a0a9452ea03991f44761a8bf (127.21.81.195:39711)
I20260809 19:25:36.560688 24005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120007ea7df34b638b49cfdcb4460385" candidate_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d635e1443c497dba9326ab5b7de3a6"
I20260809 19:25:36.560701 24150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120007ea7df34b638b49cfdcb4460385" candidate_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd900b01a0a9452ea03991f44761a8bf"
I20260809 19:25:36.560765 24150 raft_consensus.cc:3037] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:36.560784 24005 raft_consensus.cc:3037] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:36.561384 24150 raft_consensus.cc:2454] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecbb40c4b84f4804bfd5d54d340dfa05 in term 1.
I20260809 19:25:36.561504 24005 raft_consensus.cc:2454] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecbb40c4b84f4804bfd5d54d340dfa05 in term 1.
I20260809 19:25:36.561547 23791 leader_election.cc:302] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd900b01a0a9452ea03991f44761a8bf, ecbb40c4b84f4804bfd5d54d340dfa05; no voters:
I20260809 19:25:36.561686 24324 raft_consensus.cc:2781] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:36.561842 24324 raft_consensus.cc:686] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 LEADER]: Becoming Leader. State: Replica: ecbb40c4b84f4804bfd5d54d340dfa05, State: Running, Role: LEADER
I20260809 19:25:36.561930 24324 consensus_queue.cc:237] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.562713 23694 catalog_manager.cc:5495] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to ecbb40c4b84f4804bfd5d54d340dfa05 (127.21.81.193). New cstate: current_term: 1 leader_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } health_report { overall_health: UNKNOWN } } }
I20260809 19:25:36.576089 24333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:36.573451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53902 (local address 127.21.81.193:41263)
0809 19:25:36.573658 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:36.573660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:36.573668 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:36.573707 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:36.573708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:36.573726 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:36.573797 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:36.573916 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.574408 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.575250 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.575544 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.575599 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:36.575759 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:36.575867 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:36.575920 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:36.575961 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 19:25:36.575977 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":107,"threads_started":1}
I20260809 19:25:36.580935 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:36.578746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:52575 (local address 127.21.81.195:39711)
0809 19:25:36.578798 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:36.578800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:36.578831 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:25:36.578883 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:36.578884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:36.578892 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:36.578936 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:36.579021 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.579532 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.580260 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.580437 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.580489 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:36.580696 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:36.580736 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:36.580781 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:36.580841 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 19:25:36.580864 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:25:36.580966 24336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:36.578588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:39711 (local address 127.21.81.193:52575)
0809 19:25:36.578773 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:36.578787 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:36.578844 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:36.578937 (+ 93us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:36.578937 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:36.578938 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:25:36.578987 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:36.578988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.579595 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.579596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:36.580189 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:36.580191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.580511 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.580512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:36.580590 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:36.580603 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:25:36.580799 (+ 196us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:25:36.580801 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:25:36.580835 (+ 34us) client_negotiation.cc:240] Negotiation successful
0809 19:25:36.580871 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":77,"threads_started":1}
I20260809 19:25:36.583053 24005 raft_consensus.cc:1261] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Refusing update from remote peer ecbb40c4b84f4804bfd5d54d340dfa05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:36.583094 24150 raft_consensus.cc:1261] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Refusing update from remote peer ecbb40c4b84f4804bfd5d54d340dfa05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:36.583295 24324 consensus_queue.cc:1035] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:36.583423 24330 consensus_queue.cc:1035] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:36.591432 24333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:36.588785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:46197 (local address 127.21.81.193:41263)
0809 19:25:36.588839 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:36.588841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:36.589273 (+ 432us) server_negotiation.cc:407] Connection header received
0809 19:25:36.589340 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:36.589341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:36.589349 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:36.589392 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:36.589474 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.589992 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.590731 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.591018 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.591091 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:36.591231 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:36.591269 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:36.591326 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:36.591360 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 19:25:36.591379 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 19:25:36.591461 24341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:36.588746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:41263 (local address 127.21.81.195:46197)
0809 19:25:36.589209 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:36.589222 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:36.589285 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:36.589394 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:36.589394 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:36.589396 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:25:36.589440 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:36.589441 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.590058 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.590059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:36.590643 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:36.590645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:36.591093 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:36.591094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:36.591159 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:36.591164 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:25:36.591326 (+ 162us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:25:36.591327 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:25:36.591359 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 19:25:36.591401 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":103,"threads_started":1}
I20260809 19:25:36.598340 23694 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:36.598435 23694 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:36.599081 23694 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862: Sending DeleteTablet for 3 replicas of tablet 120007ea7df34b638b49cfdcb4460385
I20260809 19:25:36.599386 23985 tablet_service.cc:1514] Processing DeleteTablet for tablet 120007ea7df34b638b49cfdcb4460385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:36 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:60472
I20260809 19:25:36.599385 24129 tablet_service.cc:1514] Processing DeleteTablet for tablet 120007ea7df34b638b49cfdcb4460385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:36 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:52438
I20260809 19:25:36.599431 23838 tablet_service.cc:1514] Processing DeleteTablet for tablet 120007ea7df34b638b49cfdcb4460385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:36 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:53896
I20260809 19:25:36.599483 24307 tablet_replica.cc:330] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6: stopping tablet replica
I20260809 19:25:36.599481 24309 tablet_replica.cc:330] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf: stopping tablet replica
I20260809 19:25:36.599527 24307 raft_consensus.cc:2229] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:36.599534 24309 raft_consensus.cc:2229] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:36.599565 24307 raft_consensus.cc:2258] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:36.599572 24309 raft_consensus.cc:2258] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:36.599643 24308 tablet_replica.cc:330] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05: stopping tablet replica
I20260809 19:25:36.599699 24308 raft_consensus.cc:2229] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:25:36.599793 24308 raft_consensus.cc:2258] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:36.599776 23694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:25:36.599925 23694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:25:36.599934 24307 ts_tablet_manager.cc:1905] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:36.600530 24308 ts_tablet_manager.cc:1905] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:36.601063 23985 tablet_service.cc:1467] Processing CreateTablet for tablet f92205004fe940c98ecb80419a07a360 (DEFAULT_TABLE table=test-table [id=d7ec8e55f85543ac9933803b0586b6c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:36.601191 23985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f92205004fe940c98ecb80419a07a360. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:36.601871 24307 ts_tablet_manager.cc:1918] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:25:36.601924 24307 log.cc:1196] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/120007ea7df34b638b49cfdcb4460385
I20260809 19:25:36.602149 24307 ts_tablet_manager.cc:1939] T 120007ea7df34b638b49cfdcb4460385 P 61d635e1443c497dba9326ab5b7de3a6: Deleting consensus metadata
I20260809 19:25:36.602313 24308 ts_tablet_manager.cc:1918] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:25:36.602358 24308 log.cc:1196] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-0/wal/wals/120007ea7df34b638b49cfdcb4460385
I20260809 19:25:36.602484 23681 catalog_manager.cc:4841] TS 61d635e1443c497dba9326ab5b7de3a6 (127.21.81.194:41575): tablet 120007ea7df34b638b49cfdcb4460385 (table test-table [id=58b35875fc8847418bdcbc68d3c63a5b]) successfully deleted
I20260809 19:25:36.602551 24320 tablet_bootstrap.cc:492] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6: Bootstrap starting.
I20260809 19:25:36.602605 24308 ts_tablet_manager.cc:1939] T 120007ea7df34b638b49cfdcb4460385 P ecbb40c4b84f4804bfd5d54d340dfa05: Deleting consensus metadata
I20260809 19:25:36.602913 24320 tablet_bootstrap.cc:654] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:36.602913 23838 tablet_service.cc:1467] Processing CreateTablet for tablet f92205004fe940c98ecb80419a07a360 (DEFAULT_TABLE table=test-table [id=d7ec8e55f85543ac9933803b0586b6c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:36.603009 23838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f92205004fe940c98ecb80419a07a360. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:36.603116 24309 ts_tablet_manager.cc:1905] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:36.603529 24320 tablet_bootstrap.cc:492] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6: No bootstrap required, opened a new log
I20260809 19:25:36.603574 24320 ts_tablet_manager.cc:1397] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:36.603726 24320 raft_consensus.cc:348] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.603794 24320 raft_consensus.cc:374] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:36.603814 24320 raft_consensus.cc:729] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d635e1443c497dba9326ab5b7de3a6, State: Initialized, Role: FOLLOWER
I20260809 19:25:36.603883 24320 consensus_queue.cc:260] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.603967 24320 ts_tablet_manager.cc:1428] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:36.604260 24319 tablet_bootstrap.cc:492] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05: Bootstrap starting.
I20260809 19:25:36.604604 24309 ts_tablet_manager.cc:1918] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:25:36.604655 24319 tablet_bootstrap.cc:654] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:36.604665 24309 log.cc:1196] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/120007ea7df34b638b49cfdcb4460385
I20260809 19:25:36.604892 24309 ts_tablet_manager.cc:1939] T 120007ea7df34b638b49cfdcb4460385 P dd900b01a0a9452ea03991f44761a8bf: Deleting consensus metadata
I20260809 19:25:36.605240 23679 catalog_manager.cc:4841] TS ecbb40c4b84f4804bfd5d54d340dfa05 (127.21.81.193:41263): tablet 120007ea7df34b638b49cfdcb4460385 (table test-table [id=58b35875fc8847418bdcbc68d3c63a5b]) successfully deleted
I20260809 19:25:36.605609 23681 catalog_manager.cc:4841] TS dd900b01a0a9452ea03991f44761a8bf (127.21.81.195:39711): tablet 120007ea7df34b638b49cfdcb4460385 (table test-table [id=58b35875fc8847418bdcbc68d3c63a5b]) successfully deleted
I20260809 19:25:36.605666 24129 tablet_service.cc:1467] Processing CreateTablet for tablet f92205004fe940c98ecb80419a07a360 (DEFAULT_TABLE table=test-table [id=d7ec8e55f85543ac9933803b0586b6c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:36.605775 24129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f92205004fe940c98ecb80419a07a360. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:36.607030 24321 tablet_bootstrap.cc:492] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf: Bootstrap starting.
I20260809 19:25:36.607374 24321 tablet_bootstrap.cc:654] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:36.607590 24319 tablet_bootstrap.cc:492] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05: No bootstrap required, opened a new log
I20260809 19:25:36.607649 24319 ts_tablet_manager.cc:1397] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 19:25:36.607833 24319 raft_consensus.cc:348] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.607908 24319 raft_consensus.cc:374] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:36.607936 24319 raft_consensus.cc:729] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecbb40c4b84f4804bfd5d54d340dfa05, State: Initialized, Role: FOLLOWER
I20260809 19:25:36.607975 24319 consensus_queue.cc:260] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.607997 24321 tablet_bootstrap.cc:492] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf: No bootstrap required, opened a new log
I20260809 19:25:36.608045 24321 ts_tablet_manager.cc:1397] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:36.608096 24319 ts_tablet_manager.cc:1428] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:36.608151 24321 raft_consensus.cc:348] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.608218 24321 raft_consensus.cc:374] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:36.608253 24321 raft_consensus.cc:729] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd900b01a0a9452ea03991f44761a8bf, State: Initialized, Role: FOLLOWER
I20260809 19:25:36.608305 24321 consensus_queue.cc:260] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.608405 24321 ts_tablet_manager.cc:1428] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:36.883129 24330 raft_consensus.cc:482] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:25:36.883271 24330 raft_consensus.cc:504] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.883490 24330 leader_election.cc:288] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61d635e1443c497dba9326ab5b7de3a6 (127.21.81.194:41575), dd900b01a0a9452ea03991f44761a8bf (127.21.81.195:39711)
I20260809 19:25:36.883740 24005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f92205004fe940c98ecb80419a07a360" candidate_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d635e1443c497dba9326ab5b7de3a6" is_pre_election: true
I20260809 19:25:36.883773 24150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f92205004fe940c98ecb80419a07a360" candidate_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd900b01a0a9452ea03991f44761a8bf" is_pre_election: true
I20260809 19:25:36.883849 24005 raft_consensus.cc:2454] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecbb40c4b84f4804bfd5d54d340dfa05 in term 0.
I20260809 19:25:36.883886 24150 raft_consensus.cc:2454] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecbb40c4b84f4804bfd5d54d340dfa05 in term 0.
I20260809 19:25:36.884023 23791 leader_election.cc:302] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d635e1443c497dba9326ab5b7de3a6, ecbb40c4b84f4804bfd5d54d340dfa05; no voters:
I20260809 19:25:36.884143 24330 raft_consensus.cc:2781] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:25:36.884204 24330 raft_consensus.cc:482] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:25:36.884220 24330 raft_consensus.cc:3037] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:36.884987 24330 raft_consensus.cc:504] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.885123 24330 leader_election.cc:288] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [CANDIDATE]: Term 1 election: Requested vote from peers 61d635e1443c497dba9326ab5b7de3a6 (127.21.81.194:41575), dd900b01a0a9452ea03991f44761a8bf (127.21.81.195:39711)
I20260809 19:25:36.885291 24005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f92205004fe940c98ecb80419a07a360" candidate_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d635e1443c497dba9326ab5b7de3a6"
I20260809 19:25:36.885370 24005 raft_consensus.cc:3037] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:36.885356 24150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f92205004fe940c98ecb80419a07a360" candidate_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd900b01a0a9452ea03991f44761a8bf"
I20260809 19:25:36.885444 24150 raft_consensus.cc:3037] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:36.886116 24005 raft_consensus.cc:2454] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecbb40c4b84f4804bfd5d54d340dfa05 in term 1.
I20260809 19:25:36.886159 24150 raft_consensus.cc:2454] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecbb40c4b84f4804bfd5d54d340dfa05 in term 1.
I20260809 19:25:36.886312 23791 leader_election.cc:302] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d635e1443c497dba9326ab5b7de3a6, ecbb40c4b84f4804bfd5d54d340dfa05; no voters:
I20260809 19:25:36.886423 24330 raft_consensus.cc:2781] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:36.886480 24330 raft_consensus.cc:686] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [term 1 LEADER]: Becoming Leader. State: Replica: ecbb40c4b84f4804bfd5d54d340dfa05, State: Running, Role: LEADER
I20260809 19:25:36.886529 24330 consensus_queue.cc:237] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } }
I20260809 19:25:36.887127 23694 catalog_manager.cc:5495] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 reported cstate change: term changed from 0 to 1, leader changed from <none> to ecbb40c4b84f4804bfd5d54d340dfa05 (127.21.81.193). New cstate: current_term: 1 leader_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecbb40c4b84f4804bfd5d54d340dfa05" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 } health_report { overall_health: UNKNOWN } } }
I20260809 19:25:36.892778 24150 raft_consensus.cc:1261] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Refusing update from remote peer ecbb40c4b84f4804bfd5d54d340dfa05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:36.892802 24005 raft_consensus.cc:1261] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Refusing update from remote peer ecbb40c4b84f4804bfd5d54d340dfa05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:36.893072 24339 consensus_queue.cc:1035] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd900b01a0a9452ea03991f44761a8bf" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:36.893239 24330 consensus_queue.cc:1035] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d635e1443c497dba9326ab5b7de3a6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 19:25:36.897789 24288 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 19:25:38.415629 23695 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:38.415752 23695 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:38.416538 23695 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85447c7266364b839cdcba820608f862: Sending DeleteTablet for 3 replicas of tablet f92205004fe940c98ecb80419a07a360
I20260809 19:25:38.416806 23985 tablet_service.cc:1514] Processing DeleteTablet for tablet f92205004fe940c98ecb80419a07a360 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:38 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:60472
I20260809 19:25:38.416822 24129 tablet_service.cc:1514] Processing DeleteTablet for tablet f92205004fe940c98ecb80419a07a360 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:38 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:52438
I20260809 19:25:38.416862 23838 tablet_service.cc:1514] Processing DeleteTablet for tablet f92205004fe940c98ecb80419a07a360 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:38 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:53896
I20260809 19:25:38.416998 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 23771
I20260809 19:25:38.417047 24376 tablet_replica.cc:330] T f92205004fe940c98ecb80419a07a360 P ecbb40c4b84f4804bfd5d54d340dfa05: stopping tablet replica
I20260809 19:25:38.421993 24374 tablet_replica.cc:330] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6: stopping tablet replica
I20260809 19:25:38.422068 24374 raft_consensus.cc:2229] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:38.422123 24374 raft_consensus.cc:2258] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:38.422192 24375 tablet_replica.cc:330] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf: stopping tablet replica
I20260809 19:25:38.422330 24375 raft_consensus.cc:2229] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:38.422390 24375 raft_consensus.cc:2258] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:38.422436 24374 ts_tablet_manager.cc:1905] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:38.422693 24375 ts_tablet_manager.cc:1905] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:38.424259 24374 ts_tablet_manager.cc:1918] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 19:25:38.424338 24374 log.cc:1196] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/f92205004fe940c98ecb80419a07a360
I20260809 19:25:38.424553 24374 ts_tablet_manager.cc:1939] T f92205004fe940c98ecb80419a07a360 P 61d635e1443c497dba9326ab5b7de3a6: Deleting consensus metadata
I20260809 19:25:38.424743 24375 ts_tablet_manager.cc:1918] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 19:25:38.424863 24375 log.cc:1196] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/f92205004fe940c98ecb80419a07a360
I20260809 19:25:38.424933 23681 catalog_manager.cc:4841] TS 61d635e1443c497dba9326ab5b7de3a6 (127.21.81.194:41575): tablet f92205004fe940c98ecb80419a07a360 (table test-table [id=d7ec8e55f85543ac9933803b0586b6c2]) successfully deleted
I20260809 19:25:38.425129 24375 ts_tablet_manager.cc:1939] T f92205004fe940c98ecb80419a07a360 P dd900b01a0a9452ea03991f44761a8bf: Deleting consensus metadata
I20260809 19:25:38.425452 23681 catalog_manager.cc:4841] TS dd900b01a0a9452ea03991f44761a8bf (127.21.81.195:39711): tablet f92205004fe940c98ecb80419a07a360 (table test-table [id=d7ec8e55f85543ac9933803b0586b6c2]) successfully deleted
W20260809 19:25:38.427821 23679 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:41263 (error 108)
I20260809 19:25:38.428054 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 23916
I20260809 19:25:38.433854 24379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:38.428195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:41263 (local address 127.0.0.1:53904)
0809 19:25:38.433749 (+ 5554us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:38.433814 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:41263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5524,"thread_start_us":63,"threads_started":1}
W20260809 19:25:38.434038 23679 catalog_manager.cc:4584] TS ecbb40c4b84f4804bfd5d54d340dfa05 (127.21.81.193:41263): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f92205004fe940c98ecb80419a07a360: Network error: Client connection negotiation failed: client connection to 127.21.81.193:41263: connect: Connection refused (error 111)
I20260809 19:25:38.435839 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 24063
I20260809 19:25:38.443207 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 23661
2026-08-09T19:25:38Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6099 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:38 dist-test-slave-njxd krb5kdc[24382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:38 dist-test-slave-njxd krb5kdc[24382](info): set up 2 sockets
Aug 09 19:25:38 dist-test-slave-njxd krb5kdc[24382](info): commencing operation
krb5kdc: starting...
W20260809 19:25:39.505704 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:25:39 dist-test-slave-njxd krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:25:39.652732 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:37517
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:37517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:39.736229 24395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:39.736444 24395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:39.736469 24395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:39.737690 24395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:39.737733 24395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:39.737747 24395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:39.737761 24395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:39.737818 24395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:39.739332 24395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:37517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:37517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:39.739600 24395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:39.739850 24395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:39.745167 24404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:39.745211 24402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:39.745226 24395 server_base.cc:1034] running on GCE node
W20260809 19:25:39.745175 24401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:39.745698 24395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:39.746034 24395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:39.747202 24395 hybrid_clock.cc:648] HybridClock initialized: now 1786303539747198 us; error 41 us; skew 500 ppm
Aug 09 19:25:39 dist-test-slave-njxd krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303539, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:39.749835 24395 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:25:39.750326 24395 webserver.cc:460] Webserver started at http://127.21.81.254:38763/ using document root <none> and password file <none>
I20260809 19:25:39.750555 24395 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:39.750633 24395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:39.750782 24395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:39.751828 24395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "ca865ad275374d869e9dc8e58966cabb"
format_stamp: "Formatted at 2026-08-09 19:25:39 on dist-test-slave-njxd"
server_key: "12fd00630191e619d46e1e965c87b222"
server_key_iv: "0ff6500011d82f8afb91db4ac19e06cf"
server_key_version: "encryptionkey@0"
I20260809 19:25:39.752233 24395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "ca865ad275374d869e9dc8e58966cabb"
format_stamp: "Formatted at 2026-08-09 19:25:39 on dist-test-slave-njxd"
server_key: "12fd00630191e619d46e1e965c87b222"
server_key_iv: "0ff6500011d82f8afb91db4ac19e06cf"
server_key_version: "encryptionkey@0"
I20260809 19:25:39.753677 24395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 19:25:39.754436 24411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:39.754609 24395 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:39.754736 24395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "ca865ad275374d869e9dc8e58966cabb"
format_stamp: "Formatted at 2026-08-09 19:25:39 on dist-test-slave-njxd"
server_key: "12fd00630191e619d46e1e965c87b222"
server_key_iv: "0ff6500011d82f8afb91db4ac19e06cf"
server_key_version: "encryptionkey@0"
I20260809 19:25:39.754813 24395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:39.777077 24395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:39.779501 24395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:39.779675 24395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:39.784090 24395 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:37517
I20260809 19:25:39.784133 24466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:37517 every 8 connection(s)
I20260809 19:25:39.784509 24395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:39.785357 24467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:39.787698 24467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb: Bootstrap starting.
I20260809 19:25:39.787850 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 24395
I20260809 19:25:39.787946 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:39.788131 21831 external_mini_cluster.cc:1428] Setting key 38d72a492bbbcc33fe4434bc76ad9808
I20260809 19:25:39.788497 24467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:39.788889 24467 log.cc:824] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:39.789729 24467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb: No bootstrap required, opened a new log
I20260809 19:25:39.792104 24467 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca865ad275374d869e9dc8e58966cabb" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 37517 } }
I20260809 19:25:39.792241 24467 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:39.792264 24467 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca865ad275374d869e9dc8e58966cabb, State: Initialized, Role: FOLLOWER
I20260809 19:25:39.792412 24467 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca865ad275374d869e9dc8e58966cabb" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 37517 } }
I20260809 19:25:39.792484 24467 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:39.792506 24467 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:39.792555 24467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:39.793251 24467 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca865ad275374d869e9dc8e58966cabb" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 37517 } }
I20260809 19:25:39.793355 24467 leader_election.cc:302] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca865ad275374d869e9dc8e58966cabb; no voters:
I20260809 19:25:39.793516 24467 leader_election.cc:288] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:39.793628 24472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:39.793853 24472 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [term 1 LEADER]: Becoming Leader. State: Replica: ca865ad275374d869e9dc8e58966cabb, State: Running, Role: LEADER
I20260809 19:25:39.793918 24467 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:39.793970 24472 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca865ad275374d869e9dc8e58966cabb" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 37517 } }
I20260809 19:25:39.794324 24472 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca865ad275374d869e9dc8e58966cabb. Latest consensus state: current_term: 1 leader_uuid: "ca865ad275374d869e9dc8e58966cabb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca865ad275374d869e9dc8e58966cabb" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 37517 } } }
I20260809 19:25:39.794399 24474 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca865ad275374d869e9dc8e58966cabb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca865ad275374d869e9dc8e58966cabb" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 37517 } } }
I20260809 19:25:39.794503 24474 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:39.794476 24472 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:39.794968 24482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:39.795740 24482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 19:25:39 dist-test-slave-njxd krb5kdc[24382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:39.797271 24482 catalog_manager.cc:1347] Generated new cluster ID: 89769389c8424609a6f32d286603f65c
I20260809 19:25:39.797322 24482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:39.801129 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:39.788652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34356 (local address 127.21.81.254:37517)
0809 19:25:39.788874 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:39.788883 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:39.788915 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:25:39.789463 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:39.789472 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:39.789534 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:39.789694 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:39.790350 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:39.791852 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:39.792824 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:39.793471 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:39.793612 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:39.796991 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:39.796995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:39.796995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:39.797007 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:39.799116 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:39.799525 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:39.799528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:39.799528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:39.799643 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:39.799863 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:39.799863 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:39.799864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:39.800157 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:39.800309 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:39.800783 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 19:25:39.800922 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":102,"threads_started":1}
I20260809 19:25:39.806986 24482 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:39.807716 24482 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:39.818224 24482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca865ad275374d869e9dc8e58966cabb: Generated new TSK 0
I20260809 19:25:39.818399 24482 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:25:39.848013 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:37517
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:39.932747 24495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:39.932992 24495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:39.933035 24495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:39.934314 24495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:39.934379 24495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:39.934423 24495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:25:39.935928 24495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:37517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:39.936215 24495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:39.936456 24495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:39.941378 24500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:39.941373 24501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:39.941375 24503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:39.941706 24495 server_base.cc:1034] running on GCE node
I20260809 19:25:39.941936 24495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:39.942242 24495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:39.943403 24495 hybrid_clock.cc:648] HybridClock initialized: now 1786303539943378 us; error 59 us; skew 500 ppm
Aug 09 19:25:39 dist-test-slave-njxd krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303539, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:39.945894 24495 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:25:39.946380 24495 webserver.cc:460] Webserver started at http://127.21.81.193:41775/ using document root <none> and password file <none>
I20260809 19:25:39.946614 24495 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:39.946681 24495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:39.946815 24495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:39.947842 24495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "f8a87ded19b14742b72914800063e250"
format_stamp: "Formatted at 2026-08-09 19:25:39 on dist-test-slave-njxd"
server_key: "5f623547aa5f85de1621442757059ca3"
server_key_iv: "b2ca4ad88bc17e4cf88f11734358843a"
server_key_version: "encryptionkey@0"
I20260809 19:25:39.948211 24495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "f8a87ded19b14742b72914800063e250"
format_stamp: "Formatted at 2026-08-09 19:25:39 on dist-test-slave-njxd"
server_key: "5f623547aa5f85de1621442757059ca3"
server_key_iv: "b2ca4ad88bc17e4cf88f11734358843a"
server_key_version: "encryptionkey@0"
I20260809 19:25:39.949506 24495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:25:39.950297 24511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:39.950471 24495 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:39.950569 24495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "f8a87ded19b14742b72914800063e250"
format_stamp: "Formatted at 2026-08-09 19:25:39 on dist-test-slave-njxd"
server_key: "5f623547aa5f85de1621442757059ca3"
server_key_iv: "b2ca4ad88bc17e4cf88f11734358843a"
server_key_version: "encryptionkey@0"
I20260809 19:25:39.950759 24495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:39.966471 24495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:39.968335 24495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:39.968490 24495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:39.968699 24495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:39.969035 24495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:39.969099 24495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:39.969141 24495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:39.969168 24495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:39.975555 24495 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:32831
I20260809 19:25:39.975615 24629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:32831 every 8 connection(s)
I20260809 19:25:39.975941 24495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:25:39 dist-test-slave-njxd krb5kdc[24382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303539, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:39.983457 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 24495
I20260809 19:25:39.983562 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:25:39.983737 21831 external_mini_cluster.cc:1428] Setting key 75481f6d8075aff43c0b6e0d7d2fb689
I20260809 19:25:39.984891 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:39.976681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:57553 (local address 127.21.81.254:37517)
0809 19:25:39.976729 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:39.976731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:39.976936 (+ 205us) server_negotiation.cc:407] Connection header received
0809 19:25:39.977405 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:39.977406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:39.977415 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:39.977467 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:39.978433 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:39.978913 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:39.979830 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:39.980018 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:39.980066 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:39.982038 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:39.982040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:39.982040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:39.982047 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:39.983365 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:39.983851 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:39.983852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:39.983853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:39.983917 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:39.984234 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:39.984235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:39.984235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:39.984382 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:39.984459 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:39.984716 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 19:25:39.984800 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:25:39.985114 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:39.976648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:37517 (local address 127.21.81.193:57553)
0809 19:25:39.976808 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:39.976849 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:39.977322 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:39.977524 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:39.977527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:39.978162 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:39.978345 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:39.978351 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:39.978992 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:39.978992 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:39.979760 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:39.979762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:39.980110 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:39.980110 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:39.980205 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:39.980614 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:39.980622 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:39.981954 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:39.983450 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:39.983459 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:39.983461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:39.983772 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:39.984000 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:39.984001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:39.984001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:39.984162 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:39.984451 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:39.984458 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:39.984586 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 19:25:39.984694 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 19:25:39.984858 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260809 19:25:39.986045 24630 heartbeater.cc:347] Connected to a master server at 127.21.81.254:37517
I20260809 19:25:39.986148 24630 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:39.986361 24630 heartbeater.cc:511] Master 127.21.81.254:37517 requested a full tablet report, sending...
I20260809 19:25:39.986932 24428 ts_manager.cc:194] Registered new tserver with Master: f8a87ded19b14742b72914800063e250 (127.21.81.193:32831)
I20260809 19:25:39.987900 24428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:57553
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:25:40.012014 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:37517
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:40.093900 24635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:40.094142 24635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:40.094166 24635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:40.095412 24635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:40.095453 24635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:40.095480 24635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:25:40.096943 24635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:37517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:40.097136 24635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:40.097298 24635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:40.102480 24641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:40.102488 24643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:40.102488 24640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:40.102843 24635 server_base.cc:1034] running on GCE node
I20260809 19:25:40.103083 24635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:40.103387 24635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:40.104557 24635 hybrid_clock.cc:648] HybridClock initialized: now 1786303540104531 us; error 40 us; skew 500 ppm
Aug 09 19:25:40 dist-test-slave-njxd krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303540, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:40.107230 24635 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:25:40.107690 24635 webserver.cc:460] Webserver started at http://127.21.81.194:45999/ using document root <none> and password file <none>
I20260809 19:25:40.107920 24635 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:40.107990 24635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:40.108148 24635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:40.109180 24635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "e27ce04e7f9e435e81633eeb6600fe2f"
format_stamp: "Formatted at 2026-08-09 19:25:40 on dist-test-slave-njxd"
server_key: "49bfa40b9c17d3c5b437b155251c90b5"
server_key_iv: "7f0848269dbf83236767029ecabe807e"
server_key_version: "encryptionkey@0"
I20260809 19:25:40.109529 24635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "e27ce04e7f9e435e81633eeb6600fe2f"
format_stamp: "Formatted at 2026-08-09 19:25:40 on dist-test-slave-njxd"
server_key: "49bfa40b9c17d3c5b437b155251c90b5"
server_key_iv: "7f0848269dbf83236767029ecabe807e"
server_key_version: "encryptionkey@0"
I20260809 19:25:40.110976 24635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:40.111722 24650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:40.111892 24635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:40.111992 24635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "e27ce04e7f9e435e81633eeb6600fe2f"
format_stamp: "Formatted at 2026-08-09 19:25:40 on dist-test-slave-njxd"
server_key: "49bfa40b9c17d3c5b437b155251c90b5"
server_key_iv: "7f0848269dbf83236767029ecabe807e"
server_key_version: "encryptionkey@0"
I20260809 19:25:40.112094 24635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:40.126255 24635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:40.128121 24635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:40.128285 24635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:40.128515 24635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:40.128831 24635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:40.128886 24635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:40.128927 24635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:40.128954 24635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:40.134960 24635 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:33619
I20260809 19:25:40.135010 24771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:33619 every 8 connection(s)
I20260809 19:25:40.135331 24635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:25:40.137106 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 24635
I20260809 19:25:40.137228 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:25:40.137449 21831 external_mini_cluster.cc:1428] Setting key 63958e21b63df9ef9e1d9b7f0f36ba9f
Aug 09 19:25:40 dist-test-slave-njxd krb5kdc[24382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303540, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:40.146327 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:40.136041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:52355 (local address 127.21.81.254:37517)
0809 19:25:40.136088 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:40.136090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:40.136326 (+ 236us) server_negotiation.cc:407] Connection header received
0809 19:25:40.136786 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:40.136787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:40.136796 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:40.136852 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:40.138192 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:40.138695 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:40.140203 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:40.140512 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:40.140589 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:40.143254 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:40.143256 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:40.143256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:40.143263 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:40.144741 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:40.145262 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:40.145263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:40.145263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:40.145315 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:40.145640 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:40.145640 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:40.145641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:40.145753 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:40.145808 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:40.146186 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 19:25:40.146262 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:25:40.146603 24775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:40.136011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:37517 (local address 127.21.81.194:52355)
0809 19:25:40.136187 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:40.136231 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:40.136706 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:40.136933 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:40.136938 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:40.137842 (+ 904us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:40.138047 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:40.138053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:40.138795 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:40.138796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:40.140100 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:40.140103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:40.140600 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:40.140601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:40.140727 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:40.141296 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:40.141304 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:40.143137 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:40.144830 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:40.144839 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:40.144841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:40.145184 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:40.145385 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:40.145386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:40.145386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:40.145539 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:40.145822 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:40.145831 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:40.146037 (+ 206us) client_negotiation.cc:769] Sending connection context
0809 19:25:40.146152 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 19:25:40.146338 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
I20260809 19:25:40.147294 24772 heartbeater.cc:347] Connected to a master server at 127.21.81.254:37517
I20260809 19:25:40.147413 24772 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:40.147612 24772 heartbeater.cc:511] Master 127.21.81.254:37517 requested a full tablet report, sending...
I20260809 19:25:40.148008 24428 ts_manager.cc:194] Registered new tserver with Master: e27ce04e7f9e435e81633eeb6600fe2f (127.21.81.194:33619)
I20260809 19:25:40.148361 24428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:52355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:25:40.166491 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:37517
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:40.249692 24778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:40.249908 24778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:40.249934 24778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:40.251080 24778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:40.251113 24778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:40.251138 24778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:25:40.252629 24778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:37517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:40.252835 24778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:40.253023 24778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:40.258255 24786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:40.258252 24783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:40.258251 24784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:40.258518 24778 server_base.cc:1034] running on GCE node
I20260809 19:25:40.258735 24778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:40.258992 24778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:40.260169 24778 hybrid_clock.cc:648] HybridClock initialized: now 1786303540260145 us; error 38 us; skew 500 ppm
Aug 09 19:25:40 dist-test-slave-njxd krb5kdc[24382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303540, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:40.262902 24778 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:25:40.263370 24778 webserver.cc:460] Webserver started at http://127.21.81.195:41913/ using document root <none> and password file <none>
I20260809 19:25:40.263633 24778 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:40.263705 24778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:40.263854 24778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:40.264945 24778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "747985d73ac54d4f9c100eafbb2fe0d2"
format_stamp: "Formatted at 2026-08-09 19:25:40 on dist-test-slave-njxd"
server_key: "44c1947d1de6b01eed05a1f7b42eae2e"
server_key_iv: "370655711651776c2bcf5e1255b15874"
server_key_version: "encryptionkey@0"
I20260809 19:25:40.265314 24778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "747985d73ac54d4f9c100eafbb2fe0d2"
format_stamp: "Formatted at 2026-08-09 19:25:40 on dist-test-slave-njxd"
server_key: "44c1947d1de6b01eed05a1f7b42eae2e"
server_key_iv: "370655711651776c2bcf5e1255b15874"
server_key_version: "encryptionkey@0"
I20260809 19:25:40.267213 24778 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 19:25:40.268049 24794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:40.268242 24778 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:40.268348 24778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "747985d73ac54d4f9c100eafbb2fe0d2"
format_stamp: "Formatted at 2026-08-09 19:25:40 on dist-test-slave-njxd"
server_key: "44c1947d1de6b01eed05a1f7b42eae2e"
server_key_iv: "370655711651776c2bcf5e1255b15874"
server_key_version: "encryptionkey@0"
I20260809 19:25:40.268437 24778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:40.292315 24778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:40.294749 24778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:40.294945 24778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:40.295218 24778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:40.295644 24778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:40.295683 24778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:40.295706 24778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:40.295719 24778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:40.302243 24778 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:46175
I20260809 19:25:40.302330 24916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:46175 every 8 connection(s)
I20260809 19:25:40.302680 24778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:25:40 dist-test-slave-njxd krb5kdc[24382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303540, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:40.311882 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:40.303476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:35073 (local address 127.21.81.254:37517)
0809 19:25:40.303524 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:40.303526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:40.303769 (+ 243us) server_negotiation.cc:407] Connection header received
0809 19:25:40.304247 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:40.304248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:40.304259 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:40.304313 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:40.305257 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:40.305825 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:40.306823 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:40.307006 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:40.307055 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:40.309210 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:40.309212 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:40.309212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:40.309219 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:40.310721 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:40.311077 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:40.311078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:40.311078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:40.311130 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:40.311341 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:40.311341 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:40.311342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:40.311446 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:40.311497 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:40.311747 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 19:25:40.311813 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:25:40.312094 24919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:40.303472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:37517 (local address 127.21.81.195:35073)
0809 19:25:40.303635 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:40.303674 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:40.304162 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:40.304369 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:40.304373 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:40.304962 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:40.305171 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:40.305176 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:40.305907 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:40.305908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:40.306746 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:40.306748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:40.307079 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:40.307080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:40.307189 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:40.307698 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:40.307706 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:40.309118 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:40.310789 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:40.310795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:40.310797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:40.311012 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:40.311186 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:40.311186 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:40.311187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:40.311288 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:40.311500 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:40.311504 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:40.311636 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 19:25:40.311719 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:25:40.311843 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260809 19:25:40.312361 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 24778
I20260809 19:25:40.312470 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:25:40.312647 21831 external_mini_cluster.cc:1428] Setting key 6eebbe5737cc9a34c72f8bdd9e048404
I20260809 19:25:40.312718 24917 heartbeater.cc:347] Connected to a master server at 127.21.81.254:37517
I20260809 19:25:40.312824 24917 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:40.313043 24917 heartbeater.cc:511] Master 127.21.81.254:37517 requested a full tablet report, sending...
I20260809 19:25:40.313488 24427 ts_manager.cc:194] Registered new tserver with Master: 747985d73ac54d4f9c100eafbb2fe0d2 (127.21.81.195:46175)
I20260809 19:25:40.313903 24427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:35073
I20260809 19:25:40.324465 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 19:25:40.336988 24470 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:40.336036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34370 (local address 127.21.81.254:37517)
0809 19:25:40.336137 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:40.336139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:40.336197 (+ 58us) server_negotiation.cc:407] Connection header received
0809 19:25:40.336260 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:40.336261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:40.336271 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:40.336325 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:40.336917 (+ 592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34370: BlockingRecv error: recv got EOF from 127.0.0.1:34370 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 19:25:40.337968 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 24495
I20260809 19:25:40.345366 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 24635
I20260809 19:25:40.350939 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 24778
I20260809 19:25:40.357267 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 24395
2026-08-09T19:25:40Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1912 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:40 dist-test-slave-njxd krb5kdc[24929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:40 dist-test-slave-njxd krb5kdc[24929](info): set up 2 sockets
Aug 09 19:25:40 dist-test-slave-njxd krb5kdc[24929](info): commencing operation
krb5kdc: starting...
W20260809 19:25:41.420666 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:25:41 dist-test-slave-njxd krb5kdc[24929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:25:41.544556 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:36963
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:33461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:36963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:41.626546 24942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:41.626794 24942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:41.626832 24942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:41.627998 24942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:41.628072 24942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:41.628094 24942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:41.628115 24942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:41.628155 24942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:41.629681 24942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:33461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:36963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:36963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:41.629967 24942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:41.630208 24942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:41.635262 24948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:41.635241 24949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:41.635236 24951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:41.635471 24942 server_base.cc:1034] running on GCE node
I20260809 19:25:41.635785 24942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:41.636099 24942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:41.637286 24942 hybrid_clock.cc:648] HybridClock initialized: now 1786303541637278 us; error 40 us; skew 500 ppm
Aug 09 19:25:41 dist-test-slave-njxd krb5kdc[24929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303541, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:41.644785 24942 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:25:41.645291 24942 webserver.cc:460] Webserver started at http://127.21.81.254:36399/ using document root <none> and password file <none>
I20260809 19:25:41.645519 24942 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:41.645561 24942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:41.645735 24942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:41.646819 24942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "e7b4addd28594e0f8ffdf6afd06f75e9"
format_stamp: "Formatted at 2026-08-09 19:25:41 on dist-test-slave-njxd"
server_key: "d31abd6f1df079f491ffa156daaa4186"
server_key_iv: "ef0538aed7aa7815875dd2f38a31dbc0"
server_key_version: "encryptionkey@0"
I20260809 19:25:41.647151 24942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "e7b4addd28594e0f8ffdf6afd06f75e9"
format_stamp: "Formatted at 2026-08-09 19:25:41 on dist-test-slave-njxd"
server_key: "d31abd6f1df079f491ffa156daaa4186"
server_key_iv: "ef0538aed7aa7815875dd2f38a31dbc0"
server_key_version: "encryptionkey@0"
I20260809 19:25:41.648494 24942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:25:41.649228 24958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:41.649451 24942 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:41.649534 24942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "e7b4addd28594e0f8ffdf6afd06f75e9"
format_stamp: "Formatted at 2026-08-09 19:25:41 on dist-test-slave-njxd"
server_key: "d31abd6f1df079f491ffa156daaa4186"
server_key_iv: "ef0538aed7aa7815875dd2f38a31dbc0"
server_key_version: "encryptionkey@0"
I20260809 19:25:41.649657 24942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:41.675266 24942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:41.677357 24942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:41.677474 24942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:41.681725 24942 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:36963
I20260809 19:25:41.681820 25013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:36963 every 8 connection(s)
I20260809 19:25:41.682380 24942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:41.683130 25014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:41.685606 25014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9: Bootstrap starting.
I20260809 19:25:41.686244 25014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:41.686574 25014 log.cc:824] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:41.687307 25014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9: No bootstrap required, opened a new log
I20260809 19:25:41.689003 25014 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b4addd28594e0f8ffdf6afd06f75e9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 36963 } }
I20260809 19:25:41.689172 25014 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:41.689225 25014 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7b4addd28594e0f8ffdf6afd06f75e9, State: Initialized, Role: FOLLOWER
I20260809 19:25:41.689421 25014 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b4addd28594e0f8ffdf6afd06f75e9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 36963 } }
I20260809 19:25:41.689528 25014 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:41.689601 25014 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:41.689657 25014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:41.690410 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 24942
I20260809 19:25:41.690490 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:41.690644 25014 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b4addd28594e0f8ffdf6afd06f75e9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 36963 } }
I20260809 19:25:41.690685 21831 external_mini_cluster.cc:1428] Setting key f930974537da53debbd58b7cf0806bac
I20260809 19:25:41.690776 25014 leader_election.cc:302] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7b4addd28594e0f8ffdf6afd06f75e9; no voters:
I20260809 19:25:41.690975 25014 leader_election.cc:288] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:41.691009 25017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:41.691148 25017 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [term 1 LEADER]: Becoming Leader. State: Replica: e7b4addd28594e0f8ffdf6afd06f75e9, State: Running, Role: LEADER
I20260809 19:25:41.691270 25014 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:41.691294 25017 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b4addd28594e0f8ffdf6afd06f75e9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 36963 } }
I20260809 19:25:41.691905 25019 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7b4addd28594e0f8ffdf6afd06f75e9. Latest consensus state: current_term: 1 leader_uuid: "e7b4addd28594e0f8ffdf6afd06f75e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b4addd28594e0f8ffdf6afd06f75e9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 36963 } } }
I20260809 19:25:41.691913 25018 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7b4addd28594e0f8ffdf6afd06f75e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b4addd28594e0f8ffdf6afd06f75e9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 36963 } } }
I20260809 19:25:41.692109 25019 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:41.692142 25018 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:41.692358 25028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:41.692936 25028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:41.695649 25028 catalog_manager.cc:1347] Generated new cluster ID: 9ae0e60fd637474aa9ba195d9a5b8330
I20260809 19:25:41.695704 25028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:25:41 dist-test-slave-njxd krb5kdc[24929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:41.701997 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:41.691401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47972 (local address 127.21.81.254:36963)
0809 19:25:41.691646 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:41.691657 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:41.691692 (+ 35us) server_negotiation.cc:407] Connection header received
0809 19:25:41.691799 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:41.691804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:41.691884 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:41.692935 (+ 1051us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:41.693037 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:41.694447 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:41.695501 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:41.696069 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:41.696211 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:41.698639 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:41.698644 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:41.698644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:41.698656 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:41.700371 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:41.700777 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:41.700780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:41.700781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:41.700873 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:41.701102 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:41.701103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:41.701103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:41.701334 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:41.701516 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:41.701585 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 19:25:41.701756 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":76,"threads_started":1}
I20260809 19:25:41.702181 25028 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:41.702668 25028 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:41.703037 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:25:41.713127 25028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7b4addd28594e0f8ffdf6afd06f75e9: Generated new TSK 0
I20260809 19:25:41.713326 25028 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:25:41.720443 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:41.716616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47976 (local address 127.21.81.254:36963)
0809 19:25:41.716701 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:41.716703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:41.716848 (+ 145us) server_negotiation.cc:407] Connection header received
0809 19:25:41.716916 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:41.716916 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:41.716925 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:41.716976 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:41.717413 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:41.717968 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:41.719084 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:41.719290 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:41.719336 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:41.719987 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:41.719988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:41.719989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:41.720191 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:41.720207 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:41.720213 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:41.720259 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:41.720342 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 19:25:41.720365 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 19:25:41.723254 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 24942
2026-08-09T19:25:41Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1364 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:41 dist-test-slave-njxd krb5kdc[25048](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:41 dist-test-slave-njxd krb5kdc[25048](info): set up 2 sockets
Aug 09 19:25:41 dist-test-slave-njxd krb5kdc[25048](info): commencing operation
krb5kdc: starting...
W20260809 19:25:42.792291 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:25:42 dist-test-slave-njxd krb5kdc[25048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:25:42.914153 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:42905
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:42905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:42.996490 25061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:42.996698 25061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:42.996721 25061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:42.997841 25061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:42.997879 25061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:42.997893 25061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:42.997906 25061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:42.997918 25061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:42.999336 25061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:42905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:42905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:42.999531 25061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:42.999707 25061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:43.004683 25067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:43.004638 25066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:43.004678 25069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:43.004941 25061 server_base.cc:1034] running on GCE node
I20260809 19:25:43.005196 25061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:43.005504 25061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:43.006660 25061 hybrid_clock.cc:648] HybridClock initialized: now 1786303543006645 us; error 31 us; skew 500 ppm
Aug 09 19:25:43 dist-test-slave-njxd krb5kdc[25048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303543, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:43.009209 25061 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:25:43.009655 25061 webserver.cc:460] Webserver started at http://127.21.81.254:43051/ using document root <none> and password file <none>
I20260809 19:25:43.009851 25061 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:43.009891 25061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:43.009977 25061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:43.010968 25061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "92fbea5d12e0415b827fe8f19270f400"
format_stamp: "Formatted at 2026-08-09 19:25:43 on dist-test-slave-njxd"
server_key: "8e76fa02295925763f428223981859b8"
server_key_iv: "fd2105e93e063fc20df40f4a3a63acc5"
server_key_version: "encryptionkey@0"
I20260809 19:25:43.011277 25061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "92fbea5d12e0415b827fe8f19270f400"
format_stamp: "Formatted at 2026-08-09 19:25:43 on dist-test-slave-njxd"
server_key: "8e76fa02295925763f428223981859b8"
server_key_iv: "fd2105e93e063fc20df40f4a3a63acc5"
server_key_version: "encryptionkey@0"
I20260809 19:25:43.012528 25061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:25:43.013206 25076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:43.013384 25061 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:43.013494 25061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "92fbea5d12e0415b827fe8f19270f400"
format_stamp: "Formatted at 2026-08-09 19:25:43 on dist-test-slave-njxd"
server_key: "8e76fa02295925763f428223981859b8"
server_key_iv: "fd2105e93e063fc20df40f4a3a63acc5"
server_key_version: "encryptionkey@0"
I20260809 19:25:43.013569 25061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:43.029467 25061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:43.031258 25061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:43.031421 25061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:43.035607 25061 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:42905
I20260809 19:25:43.035645 25130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:42905 every 8 connection(s)
I20260809 19:25:43.036010 25061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:43.037075 25131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:43.038932 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 25061
I20260809 19:25:43.039040 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:43.039227 21831 external_mini_cluster.cc:1428] Setting key a45cd02803730f5c1568a809b2327392
I20260809 19:25:43.039808 25131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400: Bootstrap starting.
I20260809 19:25:43.040473 25131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:43.040808 25131 log.cc:824] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:43.041563 25131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400: No bootstrap required, opened a new log
I20260809 19:25:43.043798 25131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92fbea5d12e0415b827fe8f19270f400" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42905 } }
I20260809 19:25:43.043956 25131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:43.044004 25131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92fbea5d12e0415b827fe8f19270f400, State: Initialized, Role: FOLLOWER
I20260809 19:25:43.044183 25131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92fbea5d12e0415b827fe8f19270f400" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42905 } }
I20260809 19:25:43.044287 25131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:43.044332 25131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:43.044375 25131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:43.045061 25131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92fbea5d12e0415b827fe8f19270f400" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42905 } }
I20260809 19:25:43.045162 25131 leader_election.cc:302] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92fbea5d12e0415b827fe8f19270f400; no voters:
I20260809 19:25:43.045351 25131 leader_election.cc:288] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:43.045374 25137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:43.045488 25137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [term 1 LEADER]: Becoming Leader. State: Replica: 92fbea5d12e0415b827fe8f19270f400, State: Running, Role: LEADER
I20260809 19:25:43.045644 25131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:43.045666 25137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92fbea5d12e0415b827fe8f19270f400" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42905 } }
Aug 09 19:25:43 dist-test-slave-njxd krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:43.047785 25138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92fbea5d12e0415b827fe8f19270f400" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92fbea5d12e0415b827fe8f19270f400" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42905 } } }
W20260809 19:25:43.047866 25151 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:25:43.047932 25151 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:25:43.048199 25139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92fbea5d12e0415b827fe8f19270f400. Latest consensus state: current_term: 1 leader_uuid: "92fbea5d12e0415b827fe8f19270f400" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92fbea5d12e0415b827fe8f19270f400" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42905 } } }
I20260809 19:25:43.048282 25139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:43.047883 25138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:43.048535 25154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:43.048792 25154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:43.050518 25154 catalog_manager.cc:1347] Generated new cluster ID: 2db893c3e1584e75a78e931a85728485
I20260809 19:25:43.050570 25154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:43.051173 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:43.039752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52442 (local address 127.21.81.254:42905)
0809 19:25:43.039945 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:43.039952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:43.039976 (+ 24us) server_negotiation.cc:407] Connection header received
0809 19:25:43.040418 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:43.040423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:43.040467 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:43.040629 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:43.041438 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.042423 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.043587 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.044124 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.044233 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:43.047061 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:43.047064 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:43.047065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:43.047077 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:43.049148 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:43.049601 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:43.049610 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:43.049610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:43.049714 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:43.049958 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:43.049959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:43.049959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:43.050271 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:43.050431 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:43.050806 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 19:25:43.050951 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
I20260809 19:25:43.062352 25154 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:43.063100 25154 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:43.070101 25154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92fbea5d12e0415b827fe8f19270f400: Generated new TSK 0
I20260809 19:25:43.070307 25154 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:25:43.104293 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42905
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:43.190078 25158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:43.190316 25158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:43.190356 25158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:43.191535 25158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:43.191625 25158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:43.191672 25158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:25:43.193198 25158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:43.193481 25158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:43.193754 25158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:43.198825 25164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:43.198874 25166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:43.199035 25163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:43.199158 25158 server_base.cc:1034] running on GCE node
I20260809 19:25:43.199349 25158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:43.199584 25158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:43.200737 25158 hybrid_clock.cc:648] HybridClock initialized: now 1786303543200728 us; error 27 us; skew 500 ppm
Aug 09 19:25:43 dist-test-slave-njxd krb5kdc[25048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303543, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:43.203692 25158 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:25:43.204133 25158 webserver.cc:460] Webserver started at http://127.21.81.193:36321/ using document root <none> and password file <none>
I20260809 19:25:43.204377 25158 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:43.204442 25158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:43.204600 25158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:43.205735 25158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "84d6e781ab5940008056b69256b361ab"
format_stamp: "Formatted at 2026-08-09 19:25:43 on dist-test-slave-njxd"
server_key: "fc5371dc31a8791e6a83cf4ff2f75f25"
server_key_iv: "85dcbc43092b1e90887cda47dcdbb4d7"
server_key_version: "encryptionkey@0"
I20260809 19:25:43.206097 25158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "84d6e781ab5940008056b69256b361ab"
format_stamp: "Formatted at 2026-08-09 19:25:43 on dist-test-slave-njxd"
server_key: "fc5371dc31a8791e6a83cf4ff2f75f25"
server_key_iv: "85dcbc43092b1e90887cda47dcdbb4d7"
server_key_version: "encryptionkey@0"
I20260809 19:25:43.207435 25158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:25:43.208176 25173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:43.208338 25158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:43.208443 25158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "84d6e781ab5940008056b69256b361ab"
format_stamp: "Formatted at 2026-08-09 19:25:43 on dist-test-slave-njxd"
server_key: "fc5371dc31a8791e6a83cf4ff2f75f25"
server_key_iv: "85dcbc43092b1e90887cda47dcdbb4d7"
server_key_version: "encryptionkey@0"
I20260809 19:25:43.208518 25158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:43.229563 25158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:43.231462 25158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:43.231632 25158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:43.231863 25158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:43.232190 25158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:43.232254 25158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:43.232287 25158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:43.232314 25158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:43.238529 25158 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:40327
I20260809 19:25:43.238598 25294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:40327 every 8 connection(s)
I20260809 19:25:43.238929 25158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:25:43.240336 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 25158
I20260809 19:25:43.240437 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:25:43.240692 21831 external_mini_cluster.cc:1428] Setting key d6795bf61b82533440a9e565d8dd750f
Aug 09 19:25:43 dist-test-slave-njxd krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303543, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:43.249769 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:43.239692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:33127 (local address 127.21.81.254:42905)
0809 19:25:43.239737 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:43.239739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:43.239977 (+ 238us) server_negotiation.cc:407] Connection header received
0809 19:25:43.240601 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:43.240602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:43.240613 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:43.240660 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:43.241900 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.242412 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.243732 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.243966 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.244061 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:43.246714 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:43.246715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:43.246715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:43.246723 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:43.248237 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:43.248736 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:43.248737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:43.248738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:43.248790 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:43.249079 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:43.249079 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:43.249080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:43.249178 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:43.249229 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:43.249608 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 19:25:43.249701 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:25:43.250017 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:43.239681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42905 (local address 127.21.81.193:33127)
0809 19:25:43.239843 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:43.239877 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:43.240468 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:43.240729 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:43.240733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:43.241499 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:43.241811 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:43.241817 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.242510 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.242511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:43.243640 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:43.243643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.244079 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.244081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:43.244234 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:43.244780 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:43.244790 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:43.246622 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:43.248321 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:43.248330 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:43.248333 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:43.248655 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:43.248856 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:43.248857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:43.248857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:43.249008 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:43.249246 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:43.249253 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:43.249425 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 19:25:43.249541 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 19:25:43.249752 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":104,"threads_started":1}
I20260809 19:25:43.250627 25295 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42905
I20260809 19:25:43.250728 25295 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
I20260809 19:25:43.250944 25295 heartbeater.cc:511] Master 127.21.81.254:42905 requested a full tablet report, sending...
I20260809 19:25:43.251478 25095 ts_manager.cc:194] Registered new tserver with Master: 84d6e781ab5940008056b69256b361ab (127.21.81.193:40327)
I20260809 19:25:43.252226 25095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:33127
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:25:43.268296 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42905
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:43.350585 25303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:43.350842 25303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:43.350881 25303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:43.352102 25303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:43.352170 25303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:43.352212 25303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:25:43.353695 25303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:43.353979 25303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:43.354229 25303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:43.359432 25311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:43.359444 25303 server_base.cc:1034] running on GCE node
W20260809 19:25:43.359413 25309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:43.359444 25308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:43.359799 25303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:43.360059 25303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:43.361236 25303 hybrid_clock.cc:648] HybridClock initialized: now 1786303543361221 us; error 28 us; skew 500 ppm
Aug 09 19:25:43 dist-test-slave-njxd krb5kdc[25048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303543, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:43.363999 25303 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:25:43.364444 25303 webserver.cc:460] Webserver started at http://127.21.81.194:36379/ using document root <none> and password file <none>
I20260809 19:25:43.364668 25303 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:43.364735 25303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:43.364907 25303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:43.366058 25303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "215cf0a6f411460da9518d28f5266e70"
format_stamp: "Formatted at 2026-08-09 19:25:43 on dist-test-slave-njxd"
server_key: "8f1b478dc39badcc45617d2bb5745c5b"
server_key_iv: "4a03cfccd3b8e929db1918ec373e0c2c"
server_key_version: "encryptionkey@0"
I20260809 19:25:43.366428 25303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "215cf0a6f411460da9518d28f5266e70"
format_stamp: "Formatted at 2026-08-09 19:25:43 on dist-test-slave-njxd"
server_key: "8f1b478dc39badcc45617d2bb5745c5b"
server_key_iv: "4a03cfccd3b8e929db1918ec373e0c2c"
server_key_version: "encryptionkey@0"
I20260809 19:25:43.367763 25303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:25:43.368491 25318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:43.368664 25303 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:43.368779 25303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "215cf0a6f411460da9518d28f5266e70"
format_stamp: "Formatted at 2026-08-09 19:25:43 on dist-test-slave-njxd"
server_key: "8f1b478dc39badcc45617d2bb5745c5b"
server_key_iv: "4a03cfccd3b8e929db1918ec373e0c2c"
server_key_version: "encryptionkey@0"
I20260809 19:25:43.368853 25303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:43.375677 25303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:43.377424 25303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:43.377650 25303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:43.377892 25303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:43.378206 25303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:43.378259 25303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:43.378298 25303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:43.378324 25303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:43.384299 25303 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:43317
I20260809 19:25:43.384374 25436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:43317 every 8 connection(s)
I20260809 19:25:43.384688 25303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:25:43 dist-test-slave-njxd krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303543, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:43.393565 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 25303
I20260809 19:25:43.393725 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:25:43.393769 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:43.385438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:59477 (local address 127.21.81.254:42905)
0809 19:25:43.385510 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:43.385512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:43.385700 (+ 188us) server_negotiation.cc:407] Connection header received
0809 19:25:43.386230 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:43.386231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:43.386240 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:43.386308 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:43.387222 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.387713 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.388680 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.388992 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.389074 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:43.391286 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:43.391287 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:43.391287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:43.391295 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:43.392474 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:43.392824 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:43.392825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:43.392825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:43.392893 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:43.393099 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:43.393099 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:43.393100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:43.393195 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:43.393243 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:43.393509 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 19:25:43.393656 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 19:25:43.393852 25439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:43.385387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42905 (local address 127.21.81.194:59477)
0809 19:25:43.385544 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:43.385627 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:43.386122 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:43.386363 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:43.386366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:43.386928 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:43.387110 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:43.387114 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.387801 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.387802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:43.388587 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:43.388589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.389085 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.389086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:43.389188 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:43.389776 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:43.389790 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:43.391193 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:43.392544 (+ 1351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:43.392550 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:43.392552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:43.392760 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:43.392952 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:43.392952 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:43.392952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:43.393048 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:43.393250 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:43.393254 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:43.393358 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 19:25:43.393438 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 19:25:43.393553 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
I20260809 19:25:43.393939 21831 external_mini_cluster.cc:1428] Setting key a5316da7e9b187e66f4b57019f5e7671
I20260809 19:25:43.394380 25437 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42905
I20260809 19:25:43.394495 25437 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:43.394686 25437 heartbeater.cc:511] Master 127.21.81.254:42905 requested a full tablet report, sending...
I20260809 19:25:43.395089 25095 ts_manager.cc:194] Registered new tserver with Master: 215cf0a6f411460da9518d28f5266e70 (127.21.81.194:43317)
I20260809 19:25:43.395591 25095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:59477
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:25:43.420389 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42905
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:43.503381 25442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:43.503620 25442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:43.503659 25442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:43.504803 25442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:43.504869 25442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:43.504914 25442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:25:43.506407 25442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:43.506680 25442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:43.506911 25442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:43.511888 25447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:43.511894 25450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:43.511894 25448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:43.512328 25442 server_base.cc:1034] running on GCE node
I20260809 19:25:43.512552 25442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:43.512799 25442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:43.513965 25442 hybrid_clock.cc:648] HybridClock initialized: now 1786303543513960 us; error 36 us; skew 500 ppm
Aug 09 19:25:43 dist-test-slave-njxd krb5kdc[25048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303543, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:43.516520 25442 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:25:43.516927 25442 webserver.cc:460] Webserver started at http://127.21.81.195:45959/ using document root <none> and password file <none>
I20260809 19:25:43.517161 25442 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:43.517204 25442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:43.517347 25442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:43.518350 25442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "ce986e1e56b547d9b49dc0f80dbb3e06"
format_stamp: "Formatted at 2026-08-09 19:25:43 on dist-test-slave-njxd"
server_key: "abc672975db260fd4f871c1c01d45104"
server_key_iv: "eef681cf51fe0d0073639e5fa98e3626"
server_key_version: "encryptionkey@0"
I20260809 19:25:43.518644 25442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "ce986e1e56b547d9b49dc0f80dbb3e06"
format_stamp: "Formatted at 2026-08-09 19:25:43 on dist-test-slave-njxd"
server_key: "abc672975db260fd4f871c1c01d45104"
server_key_iv: "eef681cf51fe0d0073639e5fa98e3626"
server_key_version: "encryptionkey@0"
I20260809 19:25:43.519830 25442 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:25:43.520483 25458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:43.520649 25442 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:43.520740 25442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "ce986e1e56b547d9b49dc0f80dbb3e06"
format_stamp: "Formatted at 2026-08-09 19:25:43 on dist-test-slave-njxd"
server_key: "abc672975db260fd4f871c1c01d45104"
server_key_iv: "eef681cf51fe0d0073639e5fa98e3626"
server_key_version: "encryptionkey@0"
I20260809 19:25:43.520794 25442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:43.533838 25442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:43.535667 25442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:43.535791 25442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:43.535977 25442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:43.536293 25442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:43.536325 25442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:43.536369 25442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:43.536394 25442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:43.542377 25442 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:33241
I20260809 19:25:43.542436 25575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:33241 every 8 connection(s)
I20260809 19:25:43.542806 25442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:25:43.545703 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 25442
I20260809 19:25:43.545809 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:25:43.546038 21831 external_mini_cluster.cc:1428] Setting key 81ec58bd77984ad765ad36362bfe7b2e
Aug 09 19:25:43 dist-test-slave-njxd krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303543, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:43.552747 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:43.543549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:54637 (local address 127.21.81.254:42905)
0809 19:25:43.543636 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:43.543638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:43.543805 (+ 167us) server_negotiation.cc:407] Connection header received
0809 19:25:43.544286 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:43.544287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:43.544298 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:43.544348 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:43.545300 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.546006 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.547477 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.547687 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.547763 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:43.550022 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:43.550024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:43.550024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:43.550032 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:43.551357 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:43.551774 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:43.551776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:43.551776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:43.551839 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:43.552103 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:43.552104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:43.552104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:43.552219 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:43.552298 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:43.552572 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 19:25:43.552655 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 19:25:43.552949 25578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:43.543509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42905 (local address 127.21.81.195:54637)
0809 19:25:43.543687 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:43.543723 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:43.544206 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:43.544398 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:43.544401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:43.544983 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:43.545181 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:43.545185 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.546130 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.546131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:43.547375 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:43.547378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.547773 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.547774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:43.547909 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:43.548342 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:43.548350 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:43.549918 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:43.551441 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:43.551448 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:43.551450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:43.551680 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:43.551908 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:43.551909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:43.551910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:43.552028 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:43.552288 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:43.552295 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:43.552435 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 19:25:43.552538 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 19:25:43.552706 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260809 19:25:43.553449 25576 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42905
I20260809 19:25:43.553551 25576 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:43.553761 25576 heartbeater.cc:511] Master 127.21.81.254:42905 requested a full tablet report, sending...
I20260809 19:25:43.554137 25095 ts_manager.cc:194] Registered new tserver with Master: ce986e1e56b547d9b49dc0f80dbb3e06 (127.21.81.195:33241)
I20260809 19:25:43.554509 25095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:54637
I20260809 19:25:43.557982 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 19:25:43 dist-test-slave-njxd krb5kdc[25048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303543, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 19:25:43 dist-test-slave-njxd krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303543, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:43.582048 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:43.574696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52454 (local address 127.21.81.254:42905)
0809 19:25:43.574785 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:43.574788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:43.574894 (+ 106us) server_negotiation.cc:407] Connection header received
0809 19:25:43.575000 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:43.575001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:43.575010 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:43.575071 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:43.575691 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.576186 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.576946 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:43.577142 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:43.577219 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:43.579556 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:43.579557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:43.579558 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:43.579566 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:43.580786 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:43.581143 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:43.581144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:43.581145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:43.581206 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:43.581403 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:43.581404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:43.581404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:43.581612 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:43.581711 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:43.581848 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 19:25:43.581934 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 19:25:43.582341 25095 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52454
I20260809 19:25:43.582831 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 25158
I20260809 19:25:43.590700 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 25303
I20260809 19:25:43.599797 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 25442
I20260809 19:25:43.608632 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 25061
2026-08-09T19:25:43Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1886 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:43 dist-test-slave-njxd krb5kdc[25588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:43 dist-test-slave-njxd krb5kdc[25588](info): set up 2 sockets
Aug 09 19:25:43 dist-test-slave-njxd krb5kdc[25588](info): commencing operation
krb5kdc: starting...
W20260809 19:25:44.672043 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:25:44 dist-test-slave-njxd krb5kdc[25588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:25:44.792488 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39861
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:45041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:39861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:44.872638 25601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:44.872884 25601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:44.872921 25601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:44.874102 25601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:44.874169 25601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:44.874193 25601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:44.874214 25601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:44.874255 25601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:44.875788 25601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:45041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:39861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:44.876053 25601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:44.876274 25601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:44.881314 25611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:44.881312 25607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:44.881299 25608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:44.881510 25601 server_base.cc:1034] running on GCE node
I20260809 19:25:44.881870 25601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:44.882179 25601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:44.883345 25601 hybrid_clock.cc:648] HybridClock initialized: now 1786303544883326 us; error 49 us; skew 500 ppm
Aug 09 19:25:44 dist-test-slave-njxd krb5kdc[25588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303544, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:44.885790 25601 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:25:44.886207 25601 webserver.cc:460] Webserver started at http://127.21.81.254:43087/ using document root <none> and password file <none>
I20260809 19:25:44.886446 25601 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:44.886512 25601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:44.886670 25601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:44.887760 25601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "495cf65890ba4d73bb7a30f026c5dfa1"
format_stamp: "Formatted at 2026-08-09 19:25:44 on dist-test-slave-njxd"
server_key: "4ed298a13532156d146bc69011157425"
server_key_iv: "d583105d61fae5d7a976c44ebcd6e96d"
server_key_version: "encryptionkey@0"
I20260809 19:25:44.888150 25601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "495cf65890ba4d73bb7a30f026c5dfa1"
format_stamp: "Formatted at 2026-08-09 19:25:44 on dist-test-slave-njxd"
server_key: "4ed298a13532156d146bc69011157425"
server_key_iv: "d583105d61fae5d7a976c44ebcd6e96d"
server_key_version: "encryptionkey@0"
I20260809 19:25:44.889446 25601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:25:44.890317 25618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:44.890492 25601 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:25:44.890602 25601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "495cf65890ba4d73bb7a30f026c5dfa1"
format_stamp: "Formatted at 2026-08-09 19:25:44 on dist-test-slave-njxd"
server_key: "4ed298a13532156d146bc69011157425"
server_key_iv: "d583105d61fae5d7a976c44ebcd6e96d"
server_key_version: "encryptionkey@0"
I20260809 19:25:44.890681 25601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:44.898554 25601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:44.900264 25601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:44.900417 25601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:44.904587 25601 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:39861
I20260809 19:25:44.904637 25674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:39861 every 8 connection(s)
I20260809 19:25:44.905104 25601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:44.905902 25675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:44.907434 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 25601
I20260809 19:25:44.907522 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:44.907732 21831 external_mini_cluster.cc:1428] Setting key 64f8b28b1f183f473e41ecba3b3f5e0f
I20260809 19:25:44.908521 25675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1: Bootstrap starting.
I20260809 19:25:44.909281 25675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:44.909709 25675 log.cc:824] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:44.917984 25675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1: No bootstrap required, opened a new log
I20260809 19:25:44.920642 25675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495cf65890ba4d73bb7a30f026c5dfa1" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39861 } }
I20260809 19:25:44.920892 25675 raft_consensus.cc:374] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:44.920989 25675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 495cf65890ba4d73bb7a30f026c5dfa1, State: Initialized, Role: FOLLOWER
I20260809 19:25:44.921151 25675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495cf65890ba4d73bb7a30f026c5dfa1" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39861 } }
I20260809 19:25:44.921269 25675 raft_consensus.cc:388] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:44.921326 25675 raft_consensus.cc:482] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:44.921393 25675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:44.922158 25675 raft_consensus.cc:504] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495cf65890ba4d73bb7a30f026c5dfa1" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39861 } }
I20260809 19:25:44.922305 25675 leader_election.cc:302] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 495cf65890ba4d73bb7a30f026c5dfa1; no voters:
I20260809 19:25:44.922484 25675 leader_election.cc:288] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:44.922515 25680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:44.922655 25680 raft_consensus.cc:686] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [term 1 LEADER]: Becoming Leader. State: Replica: 495cf65890ba4d73bb7a30f026c5dfa1, State: Running, Role: LEADER
I20260809 19:25:44.922760 25680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495cf65890ba4d73bb7a30f026c5dfa1" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39861 } }
I20260809 19:25:44.922920 25675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:44.923121 25681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "495cf65890ba4d73bb7a30f026c5dfa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495cf65890ba4d73bb7a30f026c5dfa1" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39861 } } }
I20260809 19:25:44.923245 25681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:44.923336 25682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 495cf65890ba4d73bb7a30f026c5dfa1. Latest consensus state: current_term: 1 leader_uuid: "495cf65890ba4d73bb7a30f026c5dfa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495cf65890ba4d73bb7a30f026c5dfa1" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39861 } } }
I20260809 19:25:44.923491 25682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1 [sys.catalog]: This master's current role is: LEADER
Aug 09 19:25:44 dist-test-slave-njxd krb5kdc[25588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
W20260809 19:25:44.924903 25695 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:25:44.924973 25695 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:25:44.925568 25684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:44.925812 25684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:44.927572 25684 catalog_manager.cc:1347] Generated new cluster ID: 67daad6931b342d0b8fc9be85b848b8a
I20260809 19:25:44.927618 25684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:44.929276 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:44.908300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44258 (local address 127.21.81.254:39861)
0809 19:25:44.908511 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:44.908540 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:44.908575 (+ 35us) server_negotiation.cc:407] Connection header received
0809 19:25:44.909162 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:44.909170 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:44.909241 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:44.909384 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:44.910548 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:44.911938 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:44.920362 (+ 8424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:44.921099 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:44.921213 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:44.924440 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:44.924444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:44.924444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:44.924456 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:44.927584 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:44.927993 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:44.927995 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:44.927996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:44.928115 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:44.928360 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:44.928360 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:44.928361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:44.928651 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:44.928799 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:44.928940 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 19:25:44.929076 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":104,"threads_started":1}
I20260809 19:25:44.934659 25684 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:44.935375 25684 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:44.939846 25684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 495cf65890ba4d73bb7a30f026c5dfa1: Generated new TSK 0
I20260809 19:25:44.940016 25684 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:25:44.972236 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39861
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:45041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:45.092536 25701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:45.092839 25701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:45.092877 25701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:45.094666 25701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:45.094738 25701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:45.094782 25701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:25:45.097084 25701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:45041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:45.097375 25701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:45.097692 25701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:45.103001 25710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:45.103101 25707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:45.103219 25701 server_base.cc:1034] running on GCE node
W20260809 19:25:45.103013 25708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:45.103461 25701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:45.103713 25701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:45.104880 25701 hybrid_clock.cc:648] HybridClock initialized: now 1786303545104867 us; error 26 us; skew 500 ppm
Aug 09 19:25:45 dist-test-slave-njxd krb5kdc[25588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303545, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:45.107774 25701 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:25:45.108273 25701 webserver.cc:460] Webserver started at http://127.21.81.193:38645/ using document root <none> and password file <none>
I20260809 19:25:45.108548 25701 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:45.108639 25701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:45.108795 25701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:45.109948 25701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "df30a8a939e1415a8a4881f77baeb05d"
format_stamp: "Formatted at 2026-08-09 19:25:45 on dist-test-slave-njxd"
server_key: "57ee384044d0835cd85ecd1a1ef669b1"
server_key_iv: "7cf9b0cd852082cb8dff13b876ccaa61"
server_key_version: "encryptionkey@0"
I20260809 19:25:45.110363 25701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "df30a8a939e1415a8a4881f77baeb05d"
format_stamp: "Formatted at 2026-08-09 19:25:45 on dist-test-slave-njxd"
server_key: "57ee384044d0835cd85ecd1a1ef669b1"
server_key_iv: "7cf9b0cd852082cb8dff13b876ccaa61"
server_key_version: "encryptionkey@0"
I20260809 19:25:45.111776 25701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:25:45.112563 25717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:45.112771 25701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:45.112874 25701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "df30a8a939e1415a8a4881f77baeb05d"
format_stamp: "Formatted at 2026-08-09 19:25:45 on dist-test-slave-njxd"
server_key: "57ee384044d0835cd85ecd1a1ef669b1"
server_key_iv: "7cf9b0cd852082cb8dff13b876ccaa61"
server_key_version: "encryptionkey@0"
I20260809 19:25:45.112960 25701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:45.144184 25701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:45.146742 25701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:45.146925 25701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:45.147189 25701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:45.147548 25701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:45.147612 25701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:45.147650 25701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:45.147697 25701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:45.154093 25701 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:39981
I20260809 19:25:45.154178 25836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:39981 every 8 connection(s)
I20260809 19:25:45.154515 25701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:25:45.159215 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 25701
I20260809 19:25:45.159314 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:25:45.159554 21831 external_mini_cluster.cc:1428] Setting key 7dc4126a6efaa976f274e73034dc439b
Aug 09 19:25:45 dist-test-slave-njxd krb5kdc[25588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303545, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:45.165551 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:45.155340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:42415 (local address 127.21.81.254:39861)
0809 19:25:45.155417 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:45.155419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:45.155608 (+ 189us) server_negotiation.cc:407] Connection header received
0809 19:25:45.156581 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:45.156583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:45.156596 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:45.156666 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:45.157282 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.157817 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.159107 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.159392 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.159475 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:45.162275 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:45.162277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:45.162277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:45.162288 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:45.164020 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:45.164431 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:45.164432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:45.164432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:45.164499 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:45.164736 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:45.164737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:45.164737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:45.164893 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:45.164970 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:45.165319 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 19:25:45.165431 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 19:25:45.165560 25839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:45.155302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39861 (local address 127.21.81.193:42415)
0809 19:25:45.155469 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:45.155513 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:45.156449 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:45.156725 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:45.156728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:45.156990 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:45.157195 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:45.157200 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.158176 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.158177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:45.158987 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:45.159015 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.159499 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.159500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:45.159664 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:45.160321 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:45.160330 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:45.162156 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:45.164107 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:45.164120 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:45.164121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:45.164354 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:45.164565 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:45.164566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:45.164566 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:45.164666 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:45.164961 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:45.164966 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:45.165156 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 19:25:45.165245 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 19:25:45.165389 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260809 19:25:45.166477 25837 heartbeater.cc:347] Connected to a master server at 127.21.81.254:39861
I20260809 19:25:45.166596 25837 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:45.166834 25837 heartbeater.cc:511] Master 127.21.81.254:39861 requested a full tablet report, sending...
I20260809 19:25:45.167367 25634 ts_manager.cc:194] Registered new tserver with Master: df30a8a939e1415a8a4881f77baeb05d (127.21.81.193:39981)
I20260809 19:25:45.167979 25634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:42415
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:25:45.188778 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39861
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:45041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:45.271490 25842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:45.271715 25842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:45.271749 25842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:45.272953 25842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:45.272996 25842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:45.273056 25842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:25:45.274641 25842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:45041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:45.274910 25842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:45.275147 25842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:45.280380 25848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:45.280361 25847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:45.280361 25850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:45.280596 25842 server_base.cc:1034] running on GCE node
I20260809 19:25:45.280817 25842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:45.281082 25842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:45.282222 25842 hybrid_clock.cc:648] HybridClock initialized: now 1786303545282199 us; error 38 us; skew 500 ppm
Aug 09 19:25:45 dist-test-slave-njxd krb5kdc[25588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303545, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:45.284699 25842 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:25:45.285158 25842 webserver.cc:460] Webserver started at http://127.21.81.194:34447/ using document root <none> and password file <none>
I20260809 19:25:45.285364 25842 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:45.285404 25842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:45.285496 25842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:45.286568 25842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "84c1204c8a3248bd99d7572e519b9c06"
format_stamp: "Formatted at 2026-08-09 19:25:45 on dist-test-slave-njxd"
server_key: "418021ac9826a722251ed4675b2fe816"
server_key_iv: "cebcbe318deb41c6ab89af3176caf38f"
server_key_version: "encryptionkey@0"
I20260809 19:25:45.286871 25842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "84c1204c8a3248bd99d7572e519b9c06"
format_stamp: "Formatted at 2026-08-09 19:25:45 on dist-test-slave-njxd"
server_key: "418021ac9826a722251ed4675b2fe816"
server_key_iv: "cebcbe318deb41c6ab89af3176caf38f"
server_key_version: "encryptionkey@0"
I20260809 19:25:45.288131 25842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:25:45.288832 25858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:45.289006 25842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:45.289099 25842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "84c1204c8a3248bd99d7572e519b9c06"
format_stamp: "Formatted at 2026-08-09 19:25:45 on dist-test-slave-njxd"
server_key: "418021ac9826a722251ed4675b2fe816"
server_key_iv: "cebcbe318deb41c6ab89af3176caf38f"
server_key_version: "encryptionkey@0"
I20260809 19:25:45.289173 25842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:45.304759 25842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:45.306699 25842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:45.306864 25842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:45.307102 25842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:45.307427 25842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:45.307462 25842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:45.307484 25842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:45.307530 25842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:45.314128 25842 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:34519
I20260809 19:25:45.314327 25976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:34519 every 8 connection(s)
I20260809 19:25:45.314580 25842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:25:45 dist-test-slave-njxd krb5kdc[25588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303545, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:45.324481 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 25842
I20260809 19:25:45.324510 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:45.315354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:59855 (local address 127.21.81.254:39861)
0809 19:25:45.315442 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:45.315445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:45.315664 (+ 219us) server_negotiation.cc:407] Connection header received
0809 19:25:45.316645 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:45.316646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:45.316657 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:45.316712 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:45.317383 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.317972 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.318984 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.319167 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.319215 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:45.321477 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:45.321479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:45.321480 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:45.321490 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:45.323092 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:45.323526 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:45.323527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:45.323528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:45.323588 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:45.323817 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:45.323817 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:45.323817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:45.323925 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:45.323978 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:45.324362 (+ 384us) server_negotiation.cc:299] Negotiation successful
0809 19:25:45.324440 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 19:25:45.324604 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:25:45.324796 25979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:45.315311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39861 (local address 127.21.81.194:59855)
0809 19:25:45.315484 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:45.315522 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:45.316548 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:45.316829 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:45.316833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:45.317102 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:45.317303 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:45.317307 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.318058 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.318059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:45.318908 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:45.318910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.319238 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.319239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:45.319356 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:45.319855 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:45.319863 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:45.321372 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:45.323176 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:45.323184 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:45.323186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:45.323441 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:45.323655 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:45.323656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:45.323656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:45.323758 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:45.323981 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:45.323986 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:45.324143 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 19:25:45.324331 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 19:25:45.324535 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":106,"threads_started":1}
I20260809 19:25:45.324824 21831 external_mini_cluster.cc:1428] Setting key 6baa0b86b20c8d080f34fe4d7105c23c
I20260809 19:25:45.325367 25977 heartbeater.cc:347] Connected to a master server at 127.21.81.254:39861
I20260809 19:25:45.325477 25977 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:45.325711 25977 heartbeater.cc:511] Master 127.21.81.254:39861 requested a full tablet report, sending...
I20260809 19:25:45.326135 25635 ts_manager.cc:194] Registered new tserver with Master: 84c1204c8a3248bd99d7572e519b9c06 (127.21.81.194:34519)
I20260809 19:25:45.326519 25635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:59855
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:25:45.352005 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39861
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:45041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:45.438011 25982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:45.438357 25982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:45.438414 25982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:45.440094 25982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:45.440163 25982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:45.440207 25982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:25:45.442049 25982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:45041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:45.442243 25982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:45.442422 25982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:45.447364 25990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:45.447373 25987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:45.447363 25988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:45.447602 25982 server_base.cc:1034] running on GCE node
I20260809 19:25:45.447832 25982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:45.448100 25982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:45.449255 25982 hybrid_clock.cc:648] HybridClock initialized: now 1786303545449236 us; error 38 us; skew 500 ppm
Aug 09 19:25:45 dist-test-slave-njxd krb5kdc[25588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303545, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:45.451874 25982 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:25:45.452368 25982 webserver.cc:460] Webserver started at http://127.21.81.195:33029/ using document root <none> and password file <none>
I20260809 19:25:45.452600 25982 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:45.452667 25982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:45.452827 25982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:45.453972 25982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "c37f8a2b3a954aa18087bad9c643bdc4"
format_stamp: "Formatted at 2026-08-09 19:25:45 on dist-test-slave-njxd"
server_key: "8a0f340a481bcd8527bab21d65450ac4"
server_key_iv: "c32fba7e1653907e9d914f4085105a42"
server_key_version: "encryptionkey@0"
I20260809 19:25:45.454342 25982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "c37f8a2b3a954aa18087bad9c643bdc4"
format_stamp: "Formatted at 2026-08-09 19:25:45 on dist-test-slave-njxd"
server_key: "8a0f340a481bcd8527bab21d65450ac4"
server_key_iv: "c32fba7e1653907e9d914f4085105a42"
server_key_version: "encryptionkey@0"
I20260809 19:25:45.455703 25982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:25:45.456429 25997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:45.456629 25982 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:45.456725 25982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "c37f8a2b3a954aa18087bad9c643bdc4"
format_stamp: "Formatted at 2026-08-09 19:25:45 on dist-test-slave-njxd"
server_key: "8a0f340a481bcd8527bab21d65450ac4"
server_key_iv: "c32fba7e1653907e9d914f4085105a42"
server_key_version: "encryptionkey@0"
I20260809 19:25:45.456787 25982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:45.471120 25982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:45.472996 25982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:45.473117 25982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:45.473310 25982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:45.473713 25982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:45.473779 25982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:45.473825 25982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:45.473851 25982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:45.479946 25982 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:34015
I20260809 19:25:45.480041 26117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:34015 every 8 connection(s)
I20260809 19:25:45.480342 25982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:25:45 dist-test-slave-njxd krb5kdc[25588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303545, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:45.487360 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 25982
I20260809 19:25:45.487465 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:25:45.487654 21831 external_mini_cluster.cc:1428] Setting key a0251e206231e7af0d9098374f6f20ee
I20260809 19:25:45.490037 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:45.481097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:36813 (local address 127.21.81.254:39861)
0809 19:25:45.481167 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:45.481169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:45.481411 (+ 242us) server_negotiation.cc:407] Connection header received
0809 19:25:45.482458 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:45.482460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:45.482473 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:45.482544 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:45.483190 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.483706 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.484701 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.484928 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.485012 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:45.487086 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:45.487087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:45.487088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:45.487094 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:45.488633 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:45.489025 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:45.489026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:45.489026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:45.489093 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:45.489343 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:45.489344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:45.489344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:45.489465 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:45.489539 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:45.489855 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 19:25:45.489937 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 19:25:45.490218 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:45.481070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39861 (local address 127.21.81.195:36813)
0809 19:25:45.481286 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:45.481322 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:45.482331 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:45.482620 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:45.482624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:45.482881 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:45.483067 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:45.483071 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.483813 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.483814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:45.484601 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:45.484603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.485024 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.485025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:45.485153 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:45.485631 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:45.485646 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:45.487002 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:45.488714 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:45.488721 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:45.488723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:45.488945 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:45.489172 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:45.489177 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:45.489178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:45.489273 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:45.489541 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:45.489548 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:45.489709 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 19:25:45.489819 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 19:25:45.489977 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":104,"threads_started":1}
I20260809 19:25:45.490707 26118 heartbeater.cc:347] Connected to a master server at 127.21.81.254:39861
I20260809 19:25:45.490811 26118 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:45.490994 26118 heartbeater.cc:511] Master 127.21.81.254:39861 requested a full tablet report, sending...
I20260809 19:25:45.491328 25635 ts_manager.cc:194] Registered new tserver with Master: c37f8a2b3a954aa18087bad9c643bdc4 (127.21.81.195:34015)
I20260809 19:25:45.491710 25635 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:36813
I20260809 19:25:45.499363 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 19:25:45 dist-test-slave-njxd krb5kdc[25588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 19:25:45 dist-test-slave-njxd krb5kdc[25588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.193@KRBTEST.COM
I20260809 19:25:45.523583 26126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:45.515566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36418 (local address 127.21.81.193:39981)
0809 19:25:45.515767 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:45.515772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:45.515782 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:25:45.515810 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:45.515813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:45.515828 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:45.515894 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:45.516505 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.517226 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.517958 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.518270 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.518325 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:45.520579 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:45.520590 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:45.520591 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:45.520603 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:45.522213 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:45.522639 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:45.522649 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:45.522650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:45.522708 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:45.522962 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:45.522963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:45.522964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:45.523111 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:45.523275 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:45.523361 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 19:25:45.523448 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
Aug 09 19:25:45 dist-test-slave-njxd krb5kdc[25588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:45.531787 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:45.524943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44274 (local address 127.21.81.254:39861)
0809 19:25:45.524994 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:45.524996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:45.525020 (+ 24us) server_negotiation.cc:407] Connection header received
0809 19:25:45.525083 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:45.525084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:45.525093 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:45.525143 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:45.525724 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.526195 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.526916 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:45.527090 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:45.527139 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:45.529247 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:45.529248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:45.529248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:45.529255 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:45.530698 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:45.531015 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:45.531016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:45.531016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:45.531083 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:45.531278 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:45.531279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:45.531279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:45.531435 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:45.531539 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:45.531622 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 19:25:45.531683 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 19:25:45.532064 25635 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44274
I20260809 19:25:45.532433 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 25701
I20260809 19:25:45.543546 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 25842
I20260809 19:25:45.549782 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 25982
I20260809 19:25:45.555786 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 25601
2026-08-09T19:25:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1947 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:45 dist-test-slave-njxd krb5kdc[26132](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:45 dist-test-slave-njxd krb5kdc[26132](info): set up 2 sockets
Aug 09 19:25:45 dist-test-slave-njxd krb5kdc[26132](info): commencing operation
krb5kdc: starting...
W20260809 19:25:46.632944 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:25:46 dist-test-slave-njxd krb5kdc[26132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:25:46.754554 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:33505
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:46805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:33505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:46.837514 26145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:46.837785 26145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:46.837826 26145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:46.838968 26145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:46.839046 26145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:46.839069 26145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:46.839088 26145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:46.839120 26145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:46.840602 26145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:33505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:33505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:46.840885 26145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:46.841106 26145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:46.845783 26150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:46.845773 26151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:46.845773 26153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:46.845883 26145 server_base.cc:1034] running on GCE node
I20260809 19:25:46.846206 26145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:46.846527 26145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:46.847697 26145 hybrid_clock.cc:648] HybridClock initialized: now 1786303546847689 us; error 37 us; skew 500 ppm
Aug 09 19:25:46 dist-test-slave-njxd krb5kdc[26132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303546, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:46.854116 26145 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:25:46.854488 26145 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:46.854575 26145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:46.854760 26145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:46.855854 26145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "dd1ed2e739b74db998041a56e20aaf4f"
format_stamp: "Formatted at 2026-08-09 19:25:46 on dist-test-slave-njxd"
server_key: "7d38a522f6b8cf185c21f3e20b777736"
server_key_iv: "7e8249d68fe41df9f9419b0c86f24382"
server_key_version: "encryptionkey@0"
I20260809 19:25:46.856230 26145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "dd1ed2e739b74db998041a56e20aaf4f"
format_stamp: "Formatted at 2026-08-09 19:25:46 on dist-test-slave-njxd"
server_key: "7d38a522f6b8cf185c21f3e20b777736"
server_key_iv: "7e8249d68fe41df9f9419b0c86f24382"
server_key_version: "encryptionkey@0"
I20260809 19:25:46.857546 26145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:25:46.858343 26159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:46.858515 26145 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:46.858642 26145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "dd1ed2e739b74db998041a56e20aaf4f"
format_stamp: "Formatted at 2026-08-09 19:25:46 on dist-test-slave-njxd"
server_key: "7d38a522f6b8cf185c21f3e20b777736"
server_key_iv: "7e8249d68fe41df9f9419b0c86f24382"
server_key_version: "encryptionkey@0"
I20260809 19:25:46.858719 26145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:46.867602 26145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:46.869974 26145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:46.870141 26145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:46.874133 26145 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:33505
I20260809 19:25:46.874159 26213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:33505 every 8 connection(s)
I20260809 19:25:46.874469 26145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:46.875558 26214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:46.877992 26214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f: Bootstrap starting.
I20260809 19:25:46.878661 26214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:46.878906 26214 log.cc:824] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:46.879587 26214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f: No bootstrap required, opened a new log
I20260809 19:25:46.879760 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 26145
I20260809 19:25:46.879849 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:46.879978 21831 external_mini_cluster.cc:1428] Setting key 57128f08dc92e532760bd9c8215d5d1c
I20260809 19:25:46.881753 26214 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1ed2e739b74db998041a56e20aaf4f" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33505 } }
I20260809 19:25:46.881932 26214 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:46.881971 26214 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd1ed2e739b74db998041a56e20aaf4f, State: Initialized, Role: FOLLOWER
I20260809 19:25:46.882133 26214 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1ed2e739b74db998041a56e20aaf4f" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33505 } }
I20260809 19:25:46.882267 26214 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:46.882315 26214 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:46.882355 26214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:46.883037 26214 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1ed2e739b74db998041a56e20aaf4f" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33505 } }
I20260809 19:25:46.883147 26214 leader_election.cc:302] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd1ed2e739b74db998041a56e20aaf4f; no voters:
I20260809 19:25:46.883322 26214 leader_election.cc:288] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:46.883394 26219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:46.883563 26214 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:46.883569 26219 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [term 1 LEADER]: Becoming Leader. State: Replica: dd1ed2e739b74db998041a56e20aaf4f, State: Running, Role: LEADER
I20260809 19:25:46.883814 26219 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1ed2e739b74db998041a56e20aaf4f" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33505 } }
I20260809 19:25:46.884140 26219 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd1ed2e739b74db998041a56e20aaf4f. Latest consensus state: current_term: 1 leader_uuid: "dd1ed2e739b74db998041a56e20aaf4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1ed2e739b74db998041a56e20aaf4f" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33505 } } }
I20260809 19:25:46.884217 26219 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:46.884408 26229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:46.884392 26222 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd1ed2e739b74db998041a56e20aaf4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1ed2e739b74db998041a56e20aaf4f" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33505 } } }
I20260809 19:25:46.884516 26222 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:46.885076 26229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:46.886879 26229 catalog_manager.cc:1347] Generated new cluster ID: 578fc35b8663438bb4552754db28f8c3
I20260809 19:25:46.886934 26229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:25:46 dist-test-slave-njxd krb5kdc[26132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:46.891938 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:46.880502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34080 (local address 127.21.81.254:33505)
0809 19:25:46.880719 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:46.880736 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:46.880772 (+ 36us) server_negotiation.cc:407] Connection header received
0809 19:25:46.881705 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:46.881715 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:46.881790 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:46.881943 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:46.882693 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:46.884143 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:46.885369 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:46.885879 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:46.885985 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:46.888559 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:46.888562 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:46.888563 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:46.888573 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:46.890292 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:46.890694 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:46.890697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:46.890697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:46.890807 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:46.891019 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:46.891020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:46.891021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:46.891266 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:46.891438 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:46.891535 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 19:25:46.891782 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260809 19:25:46.895298 26229 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:46.895922 26229 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:46.901832 26229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f: Generated new TSK 0
I20260809 19:25:46.902061 26229 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:25:46.931933 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:33505
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:46805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:47.021306 26242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:47.021526 26242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:47.021564 26242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:47.022723 26242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:47.022791 26242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:47.022835 26242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:25:47.024439 26242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:33505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:47.024700 26242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:47.024931 26242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:47.030821 26251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:47.030817 26248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:47.030821 26249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:47.031098 26242 server_base.cc:1034] running on GCE node
I20260809 19:25:47.031309 26242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:47.031544 26242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:47.032706 26242 hybrid_clock.cc:648] HybridClock initialized: now 1786303547032683 us; error 39 us; skew 500 ppm
Aug 09 19:25:47 dist-test-slave-njxd krb5kdc[26132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:47.035247 26242 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:25:47.035636 26242 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:47.035709 26242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:47.035856 26242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:47.036902 26242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "40ac5899ec45489e9ab33438982ebe6c"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "ef48b444b711270fb2f9046e7414052f"
server_key_iv: "8b2d94963901dc2f938b5a2e9785ade2"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.037294 26242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "40ac5899ec45489e9ab33438982ebe6c"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "ef48b444b711270fb2f9046e7414052f"
server_key_iv: "8b2d94963901dc2f938b5a2e9785ade2"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.038782 26242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:25:47.039564 26257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.039763 26242 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:47.039866 26242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "40ac5899ec45489e9ab33438982ebe6c"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "ef48b444b711270fb2f9046e7414052f"
server_key_iv: "8b2d94963901dc2f938b5a2e9785ade2"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.039952 26242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:47.052973 26242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:47.055044 26242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:47.055217 26242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:47.055477 26242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:47.055802 26242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:47.055857 26242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.055919 26242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:47.055949 26242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.062304 26242 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:41233
I20260809 19:25:47.062346 26373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:41233 every 8 connection(s)
I20260809 19:25:47.062665 26242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:25:47.068248 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 26242
I20260809 19:25:47.068334 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:25:47.068526 21831 external_mini_cluster.cc:1428] Setting key c5629e6e9d3b0d2598d32e445e3e2f05
Aug 09 19:25:47 dist-test-slave-njxd krb5kdc[26132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:47.073288 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.063444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:60409 (local address 127.21.81.254:33505)
0809 19:25:47.063495 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:47.063497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:47.063686 (+ 189us) server_negotiation.cc:407] Connection header received
0809 19:25:47.064171 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:47.064172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:47.064186 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:47.064235 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:47.065130 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.065727 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.066714 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.066915 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.066962 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.069309 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:47.069310 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:47.069310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:47.069318 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:47.071285 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:47.071949 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:47.071950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:47.071951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:47.072044 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:47.072381 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:47.072383 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:47.072383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:47.072554 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:47.072649 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:47.073032 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 19:25:47.073144 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:25:47.073472 26376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.063425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33505 (local address 127.21.81.193:60409)
0809 19:25:47.063578 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:47.063618 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:47.064078 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:47.064294 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:47.064297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:47.064553 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:47.065052 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.065058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.065811 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.065812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.066640 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.066642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.066985 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.066986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.067081 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.067540 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:47.067547 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:47.069154 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:47.071387 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:47.071396 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:47.071398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:47.071858 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:47.072137 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:47.072138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:47.072139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:47.072317 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:47.072645 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:47.072652 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:47.072847 (+ 195us) client_negotiation.cc:769] Sending connection context
0809 19:25:47.072980 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 19:25:47.073178 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260809 19:25:47.074126 26374 heartbeater.cc:347] Connected to a master server at 127.21.81.254:33505
I20260809 19:25:47.074265 26374 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:47.074438 26374 heartbeater.cc:511] Master 127.21.81.254:33505 requested a full tablet report, sending...
I20260809 19:25:47.075094 26176 ts_manager.cc:194] Registered new tserver with Master: 40ac5899ec45489e9ab33438982ebe6c (127.21.81.193:41233)
I20260809 19:25:47.076081 26176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:60409
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:25:47.098287 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:33505
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:46805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:47.180126 26379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:47.180369 26379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:47.180414 26379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:47.181653 26379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:47.181720 26379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:47.181777 26379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:25:47.183430 26379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:33505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:47.183691 26379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:47.184017 26379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:47.189285 26387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:47.189306 26385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:47.189303 26379 server_base.cc:1034] running on GCE node
W20260809 19:25:47.189285 26384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:47.189713 26379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:47.189977 26379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:47.191169 26379 hybrid_clock.cc:648] HybridClock initialized: now 1786303547191148 us; error 37 us; skew 500 ppm
Aug 09 19:25:47 dist-test-slave-njxd krb5kdc[26132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:47.194469 26379 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:25:47.194900 26379 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:47.194974 26379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:47.195117 26379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:47.196182 26379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "5151454f909447a28e59ad538d3c0dad"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "23be6d04b296a777ff4f2943b59fc469"
server_key_iv: "7878756f3290eb8536295273b0f6bb34"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.196578 26379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "5151454f909447a28e59ad538d3c0dad"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "23be6d04b296a777ff4f2943b59fc469"
server_key_iv: "7878756f3290eb8536295273b0f6bb34"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.197955 26379 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:25:47.198777 26393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.198987 26379 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:25:47.199096 26379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "5151454f909447a28e59ad538d3c0dad"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "23be6d04b296a777ff4f2943b59fc469"
server_key_iv: "7878756f3290eb8536295273b0f6bb34"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.199183 26379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:47.208681 26379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:47.210641 26379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:47.210807 26379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:47.211030 26379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:47.211349 26379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:47.211404 26379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.211447 26379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:47.211475 26379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.217717 26379 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:46811
I20260809 19:25:47.217763 26513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:46811 every 8 connection(s)
I20260809 19:25:47.218058 26379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:25:47.223687 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 26379
I20260809 19:25:47.223819 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:25:47.224021 21831 external_mini_cluster.cc:1428] Setting key 0994472e98bc8d5dd56503699fb5ee43
Aug 09 19:25:47 dist-test-slave-njxd krb5kdc[26132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:47.227999 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.218903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:38271 (local address 127.21.81.254:33505)
0809 19:25:47.218964 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:47.218966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:47.219147 (+ 181us) server_negotiation.cc:407] Connection header received
0809 19:25:47.219645 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:47.219646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:47.219656 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:47.219717 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:47.220635 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.221157 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.222199 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.222411 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.222477 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.224697 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:47.224698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:47.224699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:47.224707 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:47.226504 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:47.227049 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:47.227050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:47.227050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:47.227103 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:47.227385 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:47.227385 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:47.227386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:47.227484 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:47.227534 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:47.227864 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 19:25:47.227936 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:25:47.228263 26516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.218847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33505 (local address 127.21.81.194:38271)
0809 19:25:47.219023 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:47.219067 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:47.219555 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:47.219770 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:47.219772 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:47.220026 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:47.220516 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.220521 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.221251 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.221251 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.222093 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.222095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.222495 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.222496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.222593 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.223066 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:47.223075 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:47.224594 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:47.226625 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:47.226632 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:47.226634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:47.226965 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:47.227172 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:47.227172 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:47.227173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:47.227320 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:47.227548 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:47.227555 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:47.227729 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 19:25:47.227845 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 19:25:47.227994 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260809 19:25:47.228749 26514 heartbeater.cc:347] Connected to a master server at 127.21.81.254:33505
I20260809 19:25:47.228852 26514 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:47.229035 26514 heartbeater.cc:511] Master 127.21.81.254:33505 requested a full tablet report, sending...
I20260809 19:25:47.229414 26176 ts_manager.cc:194] Registered new tserver with Master: 5151454f909447a28e59ad538d3c0dad (127.21.81.194:46811)
I20260809 19:25:47.229871 26176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:38271
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:25:47.252604 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:33505
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:46805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:47.335546 26519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:47.335799 26519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:47.335842 26519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:47.337061 26519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:47.337133 26519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:47.337186 26519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:25:47.338835 26519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:33505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:47.339169 26519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:47.339411 26519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:47.344519 26525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:47.344687 26519 server_base.cc:1034] running on GCE node
W20260809 19:25:47.344568 26524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:47.344519 26527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:47.344990 26519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:47.345247 26519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:47.346395 26519 hybrid_clock.cc:648] HybridClock initialized: now 1786303547346373 us; error 39 us; skew 500 ppm
Aug 09 19:25:47 dist-test-slave-njxd krb5kdc[26132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:47.348896 26519 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:25:47.349237 26519 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:47.349311 26519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:47.349478 26519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:47.350612 26519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "d5676ac2103932aac74e3537d921e323"
server_key_iv: "f969cf6021673087b033def5dd3a9329"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.350987 26519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "d5676ac2103932aac74e3537d921e323"
server_key_iv: "f969cf6021673087b033def5dd3a9329"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.352322 26519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:25:47.353025 26534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.353201 26519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:47.353317 26519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "d5676ac2103932aac74e3537d921e323"
server_key_iv: "f969cf6021673087b033def5dd3a9329"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.353394 26519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:47.374366 26519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:47.376361 26519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:47.376578 26519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:47.376832 26519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:47.377197 26519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:47.377257 26519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.377301 26519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:47.377326 26519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.383467 26519 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:46173
I20260809 19:25:47.383502 26654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:46173 every 8 connection(s)
I20260809 19:25:47.383791 26519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:25:47.388456 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 26519
I20260809 19:25:47.388594 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:25:47.388837 21831 external_mini_cluster.cc:1428] Setting key ff4d40e83a131880ed641f1df30bc909
Aug 09 19:25:47 dist-test-slave-njxd krb5kdc[26132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:47.392782 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.384572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:39165 (local address 127.21.81.254:33505)
0809 19:25:47.384631 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:47.384634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:47.384880 (+ 246us) server_negotiation.cc:407] Connection header received
0809 19:25:47.385480 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:47.385481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:47.385492 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:47.385617 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:47.386556 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.387041 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.387988 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.388167 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.388213 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.390441 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:47.390442 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:47.390443 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:47.390450 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:47.391649 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:47.392028 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:47.392029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:47.392029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:47.392083 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:47.392291 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:47.392291 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:47.392291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:47.392385 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:47.392431 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:47.392656 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 19:25:47.392720 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:25:47.393004 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.384580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33505 (local address 127.21.81.195:39165)
0809 19:25:47.384765 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:47.384800 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:47.385378 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:47.385689 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:47.385693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:47.385947 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:47.386474 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.386479 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.387120 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.387120 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.387920 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.387921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.388237 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.388238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.388331 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.388833 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:47.388840 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:47.390337 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:47.391718 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:47.391742 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:47.391744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:47.391974 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:47.392141 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:47.392142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:47.392142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:47.392246 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:47.392434 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:47.392439 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:47.392546 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 19:25:47.392627 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 19:25:47.392754 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260809 19:25:47.393448 26655 heartbeater.cc:347] Connected to a master server at 127.21.81.254:33505
I20260809 19:25:47.393527 26655 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:47.393724 26655 heartbeater.cc:511] Master 127.21.81.254:33505 requested a full tablet report, sending...
I20260809 19:25:47.394115 26176 ts_manager.cc:194] Registered new tserver with Master: dc452dd3b5ba44c1be3e0efc54ca08cc (127.21.81.195:46173)
I20260809 19:25:47.394491 26176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:39165
I20260809 19:25:47.400530 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:25:47.407995 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.401864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34094 (local address 127.21.81.254:33505)
0809 19:25:47.401904 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:47.401906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:47.402036 (+ 130us) server_negotiation.cc:407] Connection header received
0809 19:25:47.402121 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:47.402122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:47.402132 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:47.402180 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:47.402766 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.403321 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.404042 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.404253 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.404329 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.405496 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:47.405497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:47.405498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:47.405505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:47.406763 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:47.407146 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:47.407148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:47.407149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:47.407207 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:47.407410 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:47.407411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:47.407412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:47.407565 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:47.407678 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:47.407811 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 19:25:47.407890 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:25:47.409441 26176 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:25:47.411041 26176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:25:47.417681 26671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.413956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:41233 (local address 127.0.0.1:43758)
0809 19:25:47.414082 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:47.414091 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:47.414134 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:47.414437 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:47.414438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:47.414440 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:47.414474 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.414475 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.415605 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.415607 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.416541 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.416544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.417453 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.417453 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.417511 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.417515 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:47.417564 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:25:47.417622 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260809 19:25:47.417742 26670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.413759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43758 (local address 127.21.81.193:41233)
0809 19:25:47.414280 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:47.414284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:47.414295 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:25:47.414332 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:47.414335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:47.414349 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:47.414429 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:47.414538 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.415502 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.416640 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.417371 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.417446 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.417498 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:47.417600 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 19:25:47.417647 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":107,"threads_started":1}
I20260809 19:25:47.417867 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.413537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34326 (local address 127.21.81.195:46173)
0809 19:25:47.413960 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:47.413964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:47.413975 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:25:47.414016 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:47.414018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:47.414035 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:47.414119 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:47.414343 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.415112 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.416766 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.417256 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.417343 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.417404 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:47.417726 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 19:25:47.417772 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":118,"threads_started":1}
I20260809 19:25:47.418040 26667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.413336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38024 (local address 127.21.81.194:46811)
0809 19:25:47.413619 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:47.413624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:47.413637 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:25:47.413682 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:47.413684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:47.413700 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:47.413791 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:47.414011 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.415168 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.416712 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.417390 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.417446 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.417498 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:47.417883 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 19:25:47.417953 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":106,"threads_started":1}
I20260809 19:25:47.417821 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.413198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:46811 (local address 127.0.0.1:38024)
0809 19:25:47.413315 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:47.413357 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:47.413422 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:47.413815 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:47.413818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:47.413820 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:47.413917 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.413920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.415261 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.415262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.416638 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.416641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.417535 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.417536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.417644 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.417648 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:47.417698 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 19:25:47.417742 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":73,"thread_start_us":67,"threads_started":1}
I20260809 19:25:47.417682 26668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.413419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:46173 (local address 127.0.0.1:34326)
0809 19:25:47.413557 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:47.413613 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:47.413672 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:47.414206 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:47.414207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:47.414209 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:47.414264 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.414265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.415219 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.415220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.416638 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.416641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.417343 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.417344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.417434 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.417450 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 19:25:47.417515 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 19:25:47.417596 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":39,"thread_start_us":90,"threads_started":1}
I20260809 19:25:47.418840 26444 tablet_service.cc:1467] Processing CreateTablet for tablet 583cc0aa3de0400dbedfffda0eeb6b54 (DEFAULT_TABLE table=test-table [id=40892c1ded7245d791e415145a32541c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:47.418859 26307 tablet_service.cc:1467] Processing CreateTablet for tablet 583cc0aa3de0400dbedfffda0eeb6b54 (DEFAULT_TABLE table=test-table [id=40892c1ded7245d791e415145a32541c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:47.418852 26585 tablet_service.cc:1467] Processing CreateTablet for tablet 583cc0aa3de0400dbedfffda0eeb6b54 (DEFAULT_TABLE table=test-table [id=40892c1ded7245d791e415145a32541c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:47.419217 26444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 583cc0aa3de0400dbedfffda0eeb6b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:47.419212 26585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 583cc0aa3de0400dbedfffda0eeb6b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:47.419212 26307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 583cc0aa3de0400dbedfffda0eeb6b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:47.422061 26672 tablet_bootstrap.cc:492] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad: Bootstrap starting.
I20260809 19:25:47.422619 26673 tablet_bootstrap.cc:492] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc: Bootstrap starting.
I20260809 19:25:47.422861 26674 tablet_bootstrap.cc:492] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c: Bootstrap starting.
I20260809 19:25:47.422958 26672 tablet_bootstrap.cc:654] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:47.423323 26673 tablet_bootstrap.cc:654] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:47.423460 26674 tablet_bootstrap.cc:654] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:47.423748 26672 log.cc:824] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:47.423945 26673 log.cc:824] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:47.424083 26674 log.cc:824] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:47.424646 26672 tablet_bootstrap.cc:492] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad: No bootstrap required, opened a new log
I20260809 19:25:47.424744 26672 ts_tablet_manager.cc:1397] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:25:47.424853 26674 tablet_bootstrap.cc:492] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c: No bootstrap required, opened a new log
I20260809 19:25:47.424921 26674 ts_tablet_manager.cc:1397] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:25:47.425019 26673 tablet_bootstrap.cc:492] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc: No bootstrap required, opened a new log
I20260809 19:25:47.425086 26673 ts_tablet_manager.cc:1397] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:25:47.426353 26672 raft_consensus.cc:348] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5151454f909447a28e59ad538d3c0dad" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 46811 } } peers { permanent_uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 46173 } } peers { permanent_uuid: "40ac5899ec45489e9ab33438982ebe6c" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41233 } }
I20260809 19:25:47.426509 26672 raft_consensus.cc:374] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:47.426553 26672 raft_consensus.cc:729] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5151454f909447a28e59ad538d3c0dad, State: Initialized, Role: FOLLOWER
I20260809 19:25:47.426687 26672 consensus_queue.cc:260] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5151454f909447a28e59ad538d3c0dad" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 46811 } } peers { permanent_uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 46173 } } peers { permanent_uuid: "40ac5899ec45489e9ab33438982ebe6c" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41233 } }
I20260809 19:25:47.426975 26514 heartbeater.cc:503] Master 127.21.81.254:33505 was elected leader, sending a full tablet report...
I20260809 19:25:47.426982 26674 raft_consensus.cc:348] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5151454f909447a28e59ad538d3c0dad" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 46811 } } peers { permanent_uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 46173 } } peers { permanent_uuid: "40ac5899ec45489e9ab33438982ebe6c" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41233 } }
I20260809 19:25:47.427129 26674 raft_consensus.cc:374] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:47.427162 26674 raft_consensus.cc:729] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40ac5899ec45489e9ab33438982ebe6c, State: Initialized, Role: FOLLOWER
I20260809 19:25:47.427117 26673 raft_consensus.cc:348] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5151454f909447a28e59ad538d3c0dad" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 46811 } } peers { permanent_uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 46173 } } peers { permanent_uuid: "40ac5899ec45489e9ab33438982ebe6c" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41233 } }
I20260809 19:25:47.427246 26673 raft_consensus.cc:374] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:47.427286 26673 raft_consensus.cc:729] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc452dd3b5ba44c1be3e0efc54ca08cc, State: Initialized, Role: FOLLOWER
I20260809 19:25:47.427273 26674 consensus_queue.cc:260] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5151454f909447a28e59ad538d3c0dad" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 46811 } } peers { permanent_uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 46173 } } peers { permanent_uuid: "40ac5899ec45489e9ab33438982ebe6c" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41233 } }
I20260809 19:25:47.426985 26672 ts_tablet_manager.cc:1428] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:25:47.427378 26673 consensus_queue.cc:260] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5151454f909447a28e59ad538d3c0dad" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 46811 } } peers { permanent_uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 46173 } } peers { permanent_uuid: "40ac5899ec45489e9ab33438982ebe6c" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41233 } }
I20260809 19:25:47.427544 26674 ts_tablet_manager.cc:1428] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:25:47.427637 26673 ts_tablet_manager.cc:1428] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:25:47.427754 26374 heartbeater.cc:503] Master 127.21.81.254:33505 was elected leader, sending a full tablet report...
I20260809 19:25:47.427767 26655 heartbeater.cc:503] Master 127.21.81.254:33505 was elected leader, sending a full tablet report...
I20260809 19:25:47.444023 26680 raft_consensus.cc:482] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:25:47.444139 26680 raft_consensus.cc:504] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5151454f909447a28e59ad538d3c0dad" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 46811 } } peers { permanent_uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 46173 } } peers { permanent_uuid: "40ac5899ec45489e9ab33438982ebe6c" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41233 } }
I20260809 19:25:47.444427 26680 leader_election.cc:288] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5151454f909447a28e59ad538d3c0dad (127.21.81.194:46811), 40ac5899ec45489e9ab33438982ebe6c (127.21.81.193:41233)
I20260809 19:25:47.447702 26667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.444761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:33347 (local address 127.21.81.194:46811)
0809 19:25:47.444821 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:47.444823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:47.444834 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:25:47.444863 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:47.444864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:47.444875 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:47.444947 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:47.445135 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.445895 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.446906 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.447331 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.447391 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.447398 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:47.447580 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 19:25:47.447631 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:25:47.447696 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.444666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:46811 (local address 127.21.81.195:33347)
0809 19:25:47.444726 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:47.444739 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:47.444785 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:47.444955 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:47.444956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:47.444958 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:47.445036 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.445038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.445978 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.445979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.446808 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.446810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.447414 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.447416 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.447501 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.447505 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:47.447562 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 19:25:47.447594 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":1}
I20260809 19:25:47.447777 26670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.444910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:34661 (local address 127.21.81.193:41233)
0809 19:25:47.444959 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:47.444961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:47.445116 (+ 155us) server_negotiation.cc:407] Connection header received
0809 19:25:47.445234 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:47.445235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:47.445245 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:47.445300 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:47.445456 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.446058 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.447015 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.447387 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.447460 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.447469 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:47.447663 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 19:25:47.447704 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:25:47.447809 26681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.444796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:41233 (local address 127.21.81.195:34661)
0809 19:25:47.445036 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:47.445047 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:47.445152 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:47.445307 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:47.445307 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:47.445309 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:47.445375 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.445377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.446145 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.446146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.446930 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:47.446932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.447486 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.447487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:47.447574 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.447578 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:47.447637 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 19:25:47.447675 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":138,"threads_started":1}
I20260809 19:25:47.448302 26465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "583cc0aa3de0400dbedfffda0eeb6b54" candidate_uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5151454f909447a28e59ad538d3c0dad" is_pre_election: true
I20260809 19:25:47.448319 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "583cc0aa3de0400dbedfffda0eeb6b54" candidate_uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40ac5899ec45489e9ab33438982ebe6c" is_pre_election: true
I20260809 19:25:47.448477 26327 raft_consensus.cc:2454] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc452dd3b5ba44c1be3e0efc54ca08cc in term 0.
I20260809 19:25:47.448472 26465 raft_consensus.cc:2454] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc452dd3b5ba44c1be3e0efc54ca08cc in term 0.
I20260809 19:25:47.448729 26538 leader_election.cc:302] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40ac5899ec45489e9ab33438982ebe6c, dc452dd3b5ba44c1be3e0efc54ca08cc; no voters:
I20260809 19:25:47.448907 26680 raft_consensus.cc:2781] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:25:47.448967 26680 raft_consensus.cc:482] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:25:47.448997 26680 raft_consensus.cc:3037] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:47.449795 26680 raft_consensus.cc:504] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5151454f909447a28e59ad538d3c0dad" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 46811 } } peers { permanent_uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 46173 } } peers { permanent_uuid: "40ac5899ec45489e9ab33438982ebe6c" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41233 } }
I20260809 19:25:47.449930 26680 leader_election.cc:288] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [CANDIDATE]: Term 1 election: Requested vote from peers 5151454f909447a28e59ad538d3c0dad (127.21.81.194:46811), 40ac5899ec45489e9ab33438982ebe6c (127.21.81.193:41233)
I20260809 19:25:47.450095 26465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "583cc0aa3de0400dbedfffda0eeb6b54" candidate_uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5151454f909447a28e59ad538d3c0dad"
I20260809 19:25:47.450091 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "583cc0aa3de0400dbedfffda0eeb6b54" candidate_uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40ac5899ec45489e9ab33438982ebe6c"
I20260809 19:25:47.450171 26465 raft_consensus.cc:3037] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:47.450194 26327 raft_consensus.cc:3037] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:47.450899 26465 raft_consensus.cc:2454] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc452dd3b5ba44c1be3e0efc54ca08cc in term 1.
I20260809 19:25:47.451025 26327 raft_consensus.cc:2454] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc452dd3b5ba44c1be3e0efc54ca08cc in term 1.
I20260809 19:25:47.451084 26538 leader_election.cc:302] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5151454f909447a28e59ad538d3c0dad, dc452dd3b5ba44c1be3e0efc54ca08cc; no voters:
I20260809 19:25:47.451227 26680 raft_consensus.cc:2781] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:47.451459 26680 raft_consensus.cc:686] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [term 1 LEADER]: Becoming Leader. State: Replica: dc452dd3b5ba44c1be3e0efc54ca08cc, State: Running, Role: LEADER
I20260809 19:25:47.451565 26680 consensus_queue.cc:237] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5151454f909447a28e59ad538d3c0dad" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 46811 } } peers { permanent_uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 46173 } } peers { permanent_uuid: "40ac5899ec45489e9ab33438982ebe6c" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41233 } }
I20260809 19:25:47.452405 26175 catalog_manager.cc:5495] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc reported cstate change: term changed from 0 to 1, leader changed from <none> to dc452dd3b5ba44c1be3e0efc54ca08cc (127.21.81.195). New cstate: current_term: 1 leader_uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5151454f909447a28e59ad538d3c0dad" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 46811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc452dd3b5ba44c1be3e0efc54ca08cc" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 46173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40ac5899ec45489e9ab33438982ebe6c" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41233 } health_report { overall_health: UNKNOWN } } }
I20260809 19:25:47.462931 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.460395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34332 (local address 127.21.81.195:46173)
0809 19:25:47.460459 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:47.460461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:47.460478 (+ 17us) server_negotiation.cc:407] Connection header received
0809 19:25:47.460508 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:47.460509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:47.460524 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:47.460582 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:47.460764 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.461209 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.462084 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:47.462301 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:47.462387 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:47.462583 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:47.462695 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:47.462789 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:47.462808 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 19:25:47.462829 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:25:47.464684 26327 raft_consensus.cc:1261] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c [term 1 FOLLOWER]: Refusing update from remote peer dc452dd3b5ba44c1be3e0efc54ca08cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:47.464716 26465 raft_consensus.cc:1261] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad [term 1 FOLLOWER]: Refusing update from remote peer dc452dd3b5ba44c1be3e0efc54ca08cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:47.465034 26680 consensus_queue.cc:1035] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "40ac5899ec45489e9ab33438982ebe6c" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:47.465158 26682 consensus_queue.cc:1035] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5151454f909447a28e59ad538d3c0dad" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 46811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:47.468899 26687 mvcc.cc:205] Tried to move back new op lower bound from 7316699330412834816 to 7316699330362318848. Current Snapshot: MvccSnapshot[applied={T|T < 7316699330412834816 or (T in {7316699330412834816})}]
I20260809 19:25:47.470033 26175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:47.470135 26175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:47.470917 26175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd1ed2e739b74db998041a56e20aaf4f: Sending DeleteTablet for 3 replicas of tablet 583cc0aa3de0400dbedfffda0eeb6b54
I20260809 19:25:47.471283 26444 tablet_service.cc:1514] Processing DeleteTablet for tablet 583cc0aa3de0400dbedfffda0eeb6b54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:47 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:38024
I20260809 19:25:47.471305 26585 tablet_service.cc:1514] Processing DeleteTablet for tablet 583cc0aa3de0400dbedfffda0eeb6b54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:47 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:34326
I20260809 19:25:47.471561 26307 tablet_service.cc:1514] Processing DeleteTablet for tablet 583cc0aa3de0400dbedfffda0eeb6b54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:47 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:43758
I20260809 19:25:47.471756 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 26242
I20260809 19:25:47.471803 26697 tablet_replica.cc:330] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc: stopping tablet replica
I20260809 19:25:47.471946 26698 tablet_replica.cc:330] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c: stopping tablet replica
I20260809 19:25:47.471951 26697 raft_consensus.cc:2229] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:25:47.472046 26697 raft_consensus.cc:2258] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:47.472148 26698 raft_consensus.cc:2229] T 583cc0aa3de0400dbedfffda0eeb6b54 P 40ac5899ec45489e9ab33438982ebe6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:47.472522 26697 ts_tablet_manager.cc:1905] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:47.473986 26697 ts_tablet_manager.cc:1918] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:25:47.474059 26697 log.cc:1196] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/583cc0aa3de0400dbedfffda0eeb6b54
I20260809 19:25:47.474385 26697 ts_tablet_manager.cc:1939] T 583cc0aa3de0400dbedfffda0eeb6b54 P dc452dd3b5ba44c1be3e0efc54ca08cc: Deleting consensus metadata
I20260809 19:25:47.474777 26163 catalog_manager.cc:4841] TS dc452dd3b5ba44c1be3e0efc54ca08cc (127.21.81.195:46173): tablet 583cc0aa3de0400dbedfffda0eeb6b54 (table test-table [id=40892c1ded7245d791e415145a32541c]) successfully deleted
I20260809 19:25:47.477353 26696 tablet_replica.cc:330] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad: stopping tablet replica
I20260809 19:25:47.477499 26696 raft_consensus.cc:2229] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:47.477640 26696 raft_consensus.cc:2258] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:47.478123 26696 ts_tablet_manager.cc:1905] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:47.479324 26696 ts_tablet_manager.cc:1918] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:25:47.479417 26696 log.cc:1196] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/583cc0aa3de0400dbedfffda0eeb6b54
I20260809 19:25:47.479674 26696 ts_tablet_manager.cc:1939] T 583cc0aa3de0400dbedfffda0eeb6b54 P 5151454f909447a28e59ad538d3c0dad: Deleting consensus metadata
I20260809 19:25:47.480127 26163 catalog_manager.cc:4841] TS 5151454f909447a28e59ad538d3c0dad (127.21.81.194:46811): tablet 583cc0aa3de0400dbedfffda0eeb6b54 (table test-table [id=40892c1ded7245d791e415145a32541c]) successfully deleted
W20260809 19:25:47.486783 26163 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:41233 (error 108)
I20260809 19:25:47.487166 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 26379
I20260809 19:25:47.489398 26671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.487267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:41233 (local address 127.0.0.1:43762)
0809 19:25:47.489296 (+ 2029us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:47.489357 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:41233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2002}
W20260809 19:25:47.489609 26163 catalog_manager.cc:4584] TS 40ac5899ec45489e9ab33438982ebe6c (127.21.81.193:41233): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 583cc0aa3de0400dbedfffda0eeb6b54: Network error: Client connection negotiation failed: client connection to 127.21.81.193:41233: connect: Connection refused (error 111)
I20260809 19:25:47.497373 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 26519
I20260809 19:25:47.504278 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 26145
2026-08-09T19:25:47Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1949 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T19:25:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:47Z Disabled control of system clock
I20260809 19:25:47.535722 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:42395
--webserver_interface=127.21.81.254
--webserver_port=0
--builtin_ntp_servers=127.21.81.212:46471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:42395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 19:25:47.618736 26703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:47.618954 26703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:47.618975 26703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:47.620076 26703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:47.620112 26703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:47.620126 26703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:47.620137 26703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:47.620148 26703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:47.621490 26703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:42395
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:42395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:47.621709 26703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:47.621942 26703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:25:47.626833 26703 server_base.cc:1034] running on GCE node
W20260809 19:25:47.626801 26708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:47.626801 26709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:47.626863 26711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:47.627295 26703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:47.627621 26703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:47.628818 26703 hybrid_clock.cc:648] HybridClock initialized: now 1786303547628798 us; error 38 us; skew 500 ppm
I20260809 19:25:47.630189 26703 webserver.cc:460] Webserver started at http://127.21.81.254:37609/ using document root <none> and password file <none>
I20260809 19:25:47.630436 26703 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:47.630504 26703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:47.630662 26703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:47.631742 26703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "d83ad830355a4d76b9bf7a78c15355f2"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "2b59503234248b19382d9f42d009cd86"
server_key_iv: "24357d95fb82bc3c379a41b124ef5524"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.632114 26703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "d83ad830355a4d76b9bf7a78c15355f2"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "2b59503234248b19382d9f42d009cd86"
server_key_iv: "24357d95fb82bc3c379a41b124ef5524"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.633438 26703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:25:47.634248 26717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.634430 26703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:47.634531 26703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "d83ad830355a4d76b9bf7a78c15355f2"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "2b59503234248b19382d9f42d009cd86"
server_key_iv: "24357d95fb82bc3c379a41b124ef5524"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.634624 26703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:47.638110 26703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:47.638424 26703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:47.638576 26703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:47.642889 26703 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:42395
I20260809 19:25:47.642922 26774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:42395 every 8 connection(s)
I20260809 19:25:47.643273 26703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:47.644279 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:47.646790 26775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2: Bootstrap starting.
I20260809 19:25:47.647408 26775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:47.647727 26775 log.cc:824] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:47.648428 26775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2: No bootstrap required, opened a new log
I20260809 19:25:47.650185 26775 raft_consensus.cc:348] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d83ad830355a4d76b9bf7a78c15355f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42395 } }
I20260809 19:25:47.650329 26775 raft_consensus.cc:374] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:47.650352 26775 raft_consensus.cc:729] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83ad830355a4d76b9bf7a78c15355f2, State: Initialized, Role: FOLLOWER
I20260809 19:25:47.650357 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 26703
I20260809 19:25:47.650482 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:47.650542 26775 consensus_queue.cc:260] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d83ad830355a4d76b9bf7a78c15355f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42395 } }
I20260809 19:25:47.650665 26775 raft_consensus.cc:388] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:47.650700 21831 external_mini_cluster.cc:1428] Setting key 01737a181e0ea1331207b568fa23e7ac
I20260809 19:25:47.650714 26775 raft_consensus.cc:482] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:47.650771 26775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:47.651526 26775 raft_consensus.cc:504] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d83ad830355a4d76b9bf7a78c15355f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42395 } }
I20260809 19:25:47.651670 26775 leader_election.cc:302] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d83ad830355a4d76b9bf7a78c15355f2; no voters:
I20260809 19:25:47.651844 26775 leader_election.cc:288] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:47.651897 26780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:47.652045 26780 raft_consensus.cc:686] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [term 1 LEADER]: Becoming Leader. State: Replica: d83ad830355a4d76b9bf7a78c15355f2, State: Running, Role: LEADER
I20260809 19:25:47.652137 26775 sys_catalog.cc:564] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:47.652184 26780 consensus_queue.cc:237] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d83ad830355a4d76b9bf7a78c15355f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42395 } }
I20260809 19:25:47.652509 26780 sys_catalog.cc:455] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d83ad830355a4d76b9bf7a78c15355f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d83ad830355a4d76b9bf7a78c15355f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42395 } } }
I20260809 19:25:47.652581 26780 sys_catalog.cc:458] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:47.652712 26780 sys_catalog.cc:455] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d83ad830355a4d76b9bf7a78c15355f2. Latest consensus state: current_term: 1 leader_uuid: "d83ad830355a4d76b9bf7a78c15355f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d83ad830355a4d76b9bf7a78c15355f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42395 } } }
I20260809 19:25:47.652784 26780 sys_catalog.cc:458] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:47.652915 26785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:47.653504 26785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:47.653918 26779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.651241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37844 (local address 127.21.81.254:42395)
0809 19:25:47.651466 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:47.651474 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:47.651500 (+ 26us) server_negotiation.cc:407] Connection header received
0809 19:25:47.651604 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:47.651609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:47.651669 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:47.651866 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:47.653028 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:47.653032 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:47.653033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:47.653038 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:47.653099 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:47.653114 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:47.653162 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:47.653456 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 19:25:47.653528 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":111,"threads_started":1}
I20260809 19:25:47.655246 26785 catalog_manager.cc:1347] Generated new cluster ID: 5858a47d5bb24cbc91a2030c0647952d
I20260809 19:25:47.655301 26785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:47.661669 26785 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:47.661846 26785 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:47.682888 26785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2: Generated new TSK 0
I20260809 19:25:47.683127 26785 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:25:47.687410 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42395
--builtin_ntp_servers=127.21.81.212:46471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 19:25:47.771984 26799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:47.772192 26799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:47.772213 26799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:47.773484 26799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:47.773525 26799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:47.773552 26799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:25:47.774991 26799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:47.775211 26799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:47.775506 26799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:47.781461 26806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:47.781469 26808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:47.781666 26799 server_base.cc:1034] running on GCE node
W20260809 19:25:47.781497 26805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:47.781947 26799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:47.782217 26799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:47.783408 26799 hybrid_clock.cc:648] HybridClock initialized: now 1786303547783387 us; error 36 us; skew 500 ppm
I20260809 19:25:47.784626 26799 webserver.cc:460] Webserver started at http://127.21.81.193:38075/ using document root <none> and password file <none>
I20260809 19:25:47.784885 26799 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:47.784960 26799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:47.785112 26799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:47.786289 26799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "58d6eebbe3a34b44a4c1e5b30f046814"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "b6ae92c544108177714f903edb4d2797"
server_key_iv: "47d31621162d70fd210ead2ed69741a0"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.786741 26799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "58d6eebbe3a34b44a4c1e5b30f046814"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "b6ae92c544108177714f903edb4d2797"
server_key_iv: "47d31621162d70fd210ead2ed69741a0"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.788125 26799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:25:47.788882 26814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.789065 26799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:47.789158 26799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "58d6eebbe3a34b44a4c1e5b30f046814"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "b6ae92c544108177714f903edb4d2797"
server_key_iv: "47d31621162d70fd210ead2ed69741a0"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.789252 26799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:47.792680 26799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:47.792986 26799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:47.793141 26799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:47.793396 26799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:47.793787 26799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:47.793843 26799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.793882 26799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:47.793910 26799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.800383 26799 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:34987
I20260809 19:25:47.800436 26931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:34987 every 8 connection(s)
I20260809 19:25:47.800765 26799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:25:47.803627 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 26799
I20260809 19:25:47.803759 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:25:47.803980 26779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.801527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:46889 (local address 127.21.81.254:42395)
0809 19:25:47.801624 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:47.801627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:47.801875 (+ 248us) server_negotiation.cc:407] Connection header received
0809 19:25:47.802348 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:47.802348 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:47.802356 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:47.802415 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:47.803609 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:47.803610 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:47.803610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:47.803615 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:47.803635 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:47.803639 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:47.803749 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:47.803904 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 19:25:47.803923 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:25:47.803988 21831 external_mini_cluster.cc:1428] Setting key 9c84b8ef6e3aab5d5b65ba14f1670dbd
I20260809 19:25:47.804198 26934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.801506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42395 (local address 127.21.81.193:46889)
0809 19:25:47.801741 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:47.801786 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:47.802265 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:47.802477 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:47.802480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:47.802488 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:47.803381 (+ 893us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:25:47.803382 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:47.803437 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:25:47.803477 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:47.803754 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:47.803762 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:47.803765 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:25:47.803891 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 19:25:47.803969 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":117,"threads_started":1}
I20260809 19:25:47.804795 26932 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42395
I20260809 19:25:47.804899 26932 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:47.805054 26932 heartbeater.cc:511] Master 127.21.81.254:42395 requested a full tablet report, sending...
I20260809 19:25:47.805115 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42395
--builtin_ntp_servers=127.21.81.212:46471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 19:25:47.805758 26734 ts_manager.cc:194] Registered new tserver with Master: 58d6eebbe3a34b44a4c1e5b30f046814 (127.21.81.193:34987)
W20260809 19:25:47.886991 26935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:47.887189 26935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:47.887224 26935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:47.889014 26935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:47.889081 26935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:47.889120 26935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:25:47.890697 26935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:47.890880 26935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:47.891127 26935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:47.896306 26941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:47.896492 26935 server_base.cc:1034] running on GCE node
W20260809 19:25:47.896310 26942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:47.896310 26944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:47.896785 26935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:47.896975 26935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:47.898116 26935 hybrid_clock.cc:648] HybridClock initialized: now 1786303547898094 us; error 36 us; skew 500 ppm
I20260809 19:25:47.899380 26935 webserver.cc:460] Webserver started at http://127.21.81.194:40203/ using document root <none> and password file <none>
I20260809 19:25:47.899560 26935 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:47.899602 26935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:47.899688 26935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:47.900799 26935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "413f356d0519462e94edbb0e204295eb"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "fa9e7053b745a0a9d148fcc1d773f23e"
server_key_iv: "a35adbd4ba927923d1a1cae50e79e8b4"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.901099 26935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "413f356d0519462e94edbb0e204295eb"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "fa9e7053b745a0a9d148fcc1d773f23e"
server_key_iv: "a35adbd4ba927923d1a1cae50e79e8b4"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.902483 26935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:25:47.903206 26950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.903381 26935 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:47.903469 26935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "413f356d0519462e94edbb0e204295eb"
format_stamp: "Formatted at 2026-08-09 19:25:47 on dist-test-slave-njxd"
server_key: "fa9e7053b745a0a9d148fcc1d773f23e"
server_key_iv: "a35adbd4ba927923d1a1cae50e79e8b4"
server_key_version: "encryptionkey@0"
I20260809 19:25:47.903525 26935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:47.906847 26935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:47.907059 26935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:47.907161 26935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:47.907317 26935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:47.907572 26935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:47.907603 26935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.907622 26935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:47.907635 26935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:47.913183 26935 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:39053
I20260809 19:25:47.913269 27068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:39053 every 8 connection(s)
I20260809 19:25:47.913666 26935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:25:47.916539 26779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.914449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:35217 (local address 127.21.81.254:42395)
0809 19:25:47.914544 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:47.914546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:47.914795 (+ 249us) server_negotiation.cc:407] Connection header received
0809 19:25:47.915228 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:47.915229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:47.915235 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:47.915295 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:47.916350 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:47.916351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:47.916352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:47.916355 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:47.916370 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:47.916374 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:47.916426 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:47.916473 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 19:25:47.916485 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 19:25:47.916761 27071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:47.914425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42395 (local address 127.21.81.194:35217)
0809 19:25:47.914646 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:47.914687 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:47.915142 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:47.915345 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:47.915348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:47.915355 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:47.916207 (+ 852us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:25:47.916207 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:47.916241 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:25:47.916268 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:47.916426 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:47.916428 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:47.916430 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:25:47.916479 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:25:47.916523 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":113,"threads_started":1}
I20260809 19:25:47.917245 27069 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42395
I20260809 19:25:47.917361 27069 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:47.917506 27069 heartbeater.cc:511] Master 127.21.81.254:42395 requested a full tablet report, sending...
I20260809 19:25:47.917881 26734 ts_manager.cc:194] Registered new tserver with Master: 413f356d0519462e94edbb0e204295eb (127.21.81.194:39053)
I20260809 19:25:47.920555 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 26935
I20260809 19:25:47.920673 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:25:47.920866 21831 external_mini_cluster.cc:1428] Setting key d0b45a799d6f8a83fb62d6ebfd59d814
I20260809 19:25:47.921912 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42395
--builtin_ntp_servers=127.21.81.212:46471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 19:25:48.004490 27072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:48.004710 27072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:48.004733 27072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:48.005849 27072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:48.005885 27072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:48.005910 27072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:25:48.007418 27072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:48.007635 27072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:48.007881 27072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:48.012874 27077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:48.012903 27078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:48.012892 27080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:48.013096 27072 server_base.cc:1034] running on GCE node
I20260809 19:25:48.013325 27072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:48.013676 27072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:48.014856 27072 hybrid_clock.cc:648] HybridClock initialized: now 1786303548014834 us; error 35 us; skew 500 ppm
I20260809 19:25:48.016113 27072 webserver.cc:460] Webserver started at http://127.21.81.195:36573/ using document root <none> and password file <none>
I20260809 19:25:48.016378 27072 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:48.016449 27072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:48.016613 27072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:48.017808 27072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "fe8415febdb24033ac8e8b5b0989eb65"
format_stamp: "Formatted at 2026-08-09 19:25:48 on dist-test-slave-njxd"
server_key: "41d0b8df3e8e982a6f23c81d1a5c004e"
server_key_iv: "0a11459f70bf44deb0a37bccc951894b"
server_key_version: "encryptionkey@0"
I20260809 19:25:48.018169 27072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "fe8415febdb24033ac8e8b5b0989eb65"
format_stamp: "Formatted at 2026-08-09 19:25:48 on dist-test-slave-njxd"
server_key: "41d0b8df3e8e982a6f23c81d1a5c004e"
server_key_iv: "0a11459f70bf44deb0a37bccc951894b"
server_key_version: "encryptionkey@0"
I20260809 19:25:48.019493 27072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:25:48.020205 27087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:48.020383 27072 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:48.020486 27072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "fe8415febdb24033ac8e8b5b0989eb65"
format_stamp: "Formatted at 2026-08-09 19:25:48 on dist-test-slave-njxd"
server_key: "41d0b8df3e8e982a6f23c81d1a5c004e"
server_key_iv: "0a11459f70bf44deb0a37bccc951894b"
server_key_version: "encryptionkey@0"
I20260809 19:25:48.020574 27072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:48.023952 27072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:48.024225 27072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:48.024370 27072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:48.024577 27072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:48.024899 27072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:48.024959 27072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:48.024999 27072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:48.025025 27072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:48.031117 27072 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:38963
I20260809 19:25:48.031205 27204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:38963 every 8 connection(s)
I20260809 19:25:48.031563 27072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:25:48.034858 26779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.032311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:52447 (local address 127.21.81.254:42395)
0809 19:25:48.032357 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:48.032360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:48.032602 (+ 242us) server_negotiation.cc:407] Connection header received
0809 19:25:48.033140 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:48.033141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:48.033148 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:48.033209 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:48.034518 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:48.034519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:48.034520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:48.034525 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:48.034540 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:48.034545 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:48.034664 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:48.034765 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 19:25:48.034784 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:25:48.035059 27207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.032283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42395 (local address 127.21.81.195:52447)
0809 19:25:48.032454 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:48.032488 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:48.033021 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:48.033285 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:48.033289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:48.033299 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:48.034350 (+ 1051us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:25:48.034351 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:48.034380 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:25:48.034410 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:48.034664 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:48.034671 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:48.034675 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:48.034774 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 19:25:48.034839 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
I20260809 19:25:48.035604 27205 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42395
I20260809 19:25:48.035691 27205 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:48.035812 27205 heartbeater.cc:511] Master 127.21.81.254:42395 requested a full tablet report, sending...
I20260809 19:25:48.036151 26734 ts_manager.cc:194] Registered new tserver with Master: fe8415febdb24033ac8e8b5b0989eb65 (127.21.81.195:38963)
I20260809 19:25:48.037410 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 27072
I20260809 19:25:48.037531 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:25:48.037760 21831 external_mini_cluster.cc:1428] Setting key 6bfa92f514a4b2004509e23730762a64
I20260809 19:25:48.038743 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:25:48.041165 26779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.040077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37852 (local address 127.21.81.254:42395)
0809 19:25:48.040145 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:48.040147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:48.040306 (+ 159us) server_negotiation.cc:407] Connection header received
0809 19:25:48.040347 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:48.040347 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:48.040352 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:48.040401 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:48.040936 (+ 535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:48.040937 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:48.040937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:48.040941 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:48.040957 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:48.040961 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:48.041005 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:48.041053 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 19:25:48.041066 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:25:48.042758 26734 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:25:48.044269 26734 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:25:48.048046 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.046439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:38963 (local address 127.0.0.1:41578)
0809 19:25:48.046561 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:48.046601 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:48.046677 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:48.047098 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:48.047105 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:48.047111 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:48.047555 (+ 444us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:25:48.047556 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:48.047572 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:25:48.047598 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:48.047860 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:48.047867 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:48.047870 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:25:48.047940 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 19:25:48.047957 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":77,"threads_started":1}
I20260809 19:25:48.048084 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.046745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41578 (local address 127.21.81.195:38963)
0809 19:25:48.046926 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:48.046932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:48.046944 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:25:48.046970 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:48.046972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:48.046983 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:48.047067 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:48.047692 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:48.047698 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:48.047699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:48.047703 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:48.047746 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:48.047770 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:48.047828 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:48.047972 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 19:25:48.047990 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260809 19:25:48.048856 27215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.046463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:39053 (local address 127.0.0.1:40402)
0809 19:25:48.047067 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:48.047077 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:48.047113 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:48.047627 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:48.047628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:48.047631 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:48.048433 (+ 802us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:25:48.048434 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:48.048441 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:25:48.048457 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:48.048704 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:48.048705 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:48.048706 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:25:48.048753 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:25:48.048766 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":102,"threads_started":1}
I20260809 19:25:48.048779 27138 tablet_service.cc:1467] Processing CreateTablet for tablet a558c4904e0e4c02b33b73e93f4d245a (DEFAULT_TABLE table=test-table [id=9ab229ac330d4ac68070cdce0687389f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:48.049142 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.046742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40402 (local address 127.21.81.194:39053)
0809 19:25:48.046981 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:48.046985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:48.047171 (+ 186us) server_negotiation.cc:407] Connection header received
0809 19:25:48.047204 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:48.047207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:48.047218 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:48.047292 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:48.048530 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:48.048538 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:48.048538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:48.048543 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:48.048586 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:48.048626 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:48.048698 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:48.049033 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 19:25:48.049052 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":120,"threads_started":1}
I20260809 19:25:48.049202 27138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a558c4904e0e4c02b33b73e93f4d245a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:48.049474 27214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.046461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:34987 (local address 127.0.0.1:54324)
0809 19:25:48.046924 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:48.046934 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:48.046984 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:48.048153 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:48.048154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:48.048156 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:48.048747 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:25:48.048748 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:48.048754 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:25:48.048767 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:48.049353 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:48.049354 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:48.049354 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 19:25:48.049399 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:25:48.049412 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":99,"threads_started":1}
I20260809 19:25:48.049716 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.046722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54324 (local address 127.21.81.193:34987)
0809 19:25:48.047689 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:48.047694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:48.047704 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:25:48.047731 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:48.047733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:48.047743 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:48.047840 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:48.048859 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:48.048871 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:48.048872 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:48.048877 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:48.049213 (+ 336us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:48.049256 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:48.049324 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:48.049602 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 19:25:48.049622 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":899,"thread_start_us":107,"threads_started":1}
I20260809 19:25:48.049988 26998 tablet_service.cc:1467] Processing CreateTablet for tablet a558c4904e0e4c02b33b73e93f4d245a (DEFAULT_TABLE table=test-table [id=9ab229ac330d4ac68070cdce0687389f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:48.050338 26998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a558c4904e0e4c02b33b73e93f4d245a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:48.052207 27219 tablet_bootstrap.cc:492] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65: Bootstrap starting.
I20260809 19:25:48.052877 27219 tablet_bootstrap.cc:654] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:48.053179 27219 log.cc:824] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:48.053334 27220 tablet_bootstrap.cc:492] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb: Bootstrap starting.
I20260809 19:25:48.053985 27220 tablet_bootstrap.cc:654] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:48.054015 26863 tablet_service.cc:1467] Processing CreateTablet for tablet a558c4904e0e4c02b33b73e93f4d245a (DEFAULT_TABLE table=test-table [id=9ab229ac330d4ac68070cdce0687389f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:48.054349 27220 log.cc:824] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:48.054385 26863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a558c4904e0e4c02b33b73e93f4d245a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:48.055073 27220 tablet_bootstrap.cc:492] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb: No bootstrap required, opened a new log
I20260809 19:25:48.055130 27220 ts_tablet_manager.cc:1397] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:25:48.056836 27223 tablet_bootstrap.cc:492] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814: Bootstrap starting.
I20260809 19:25:48.057165 27219 tablet_bootstrap.cc:492] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65: No bootstrap required, opened a new log
I20260809 19:25:48.057250 27219 ts_tablet_manager.cc:1397] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 19:25:48.057515 27223 tablet_bootstrap.cc:654] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:48.057395 27220 raft_consensus.cc:348] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe8415febdb24033ac8e8b5b0989eb65" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 38963 } } peers { permanent_uuid: "413f356d0519462e94edbb0e204295eb" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39053 } } peers { permanent_uuid: "58d6eebbe3a34b44a4c1e5b30f046814" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 34987 } }
I20260809 19:25:48.057565 27220 raft_consensus.cc:374] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:48.057657 27220 raft_consensus.cc:729] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413f356d0519462e94edbb0e204295eb, State: Initialized, Role: FOLLOWER
I20260809 19:25:48.057812 27220 consensus_queue.cc:260] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe8415febdb24033ac8e8b5b0989eb65" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 38963 } } peers { permanent_uuid: "413f356d0519462e94edbb0e204295eb" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39053 } } peers { permanent_uuid: "58d6eebbe3a34b44a4c1e5b30f046814" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 34987 } }
I20260809 19:25:48.057902 27223 log.cc:824] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:48.058108 27220 ts_tablet_manager.cc:1428] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 19:25:48.058230 27069 heartbeater.cc:503] Master 127.21.81.254:42395 was elected leader, sending a full tablet report...
I20260809 19:25:48.058707 27223 tablet_bootstrap.cc:492] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814: No bootstrap required, opened a new log
I20260809 19:25:48.058780 27223 ts_tablet_manager.cc:1397] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:25:48.059984 27219 raft_consensus.cc:348] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe8415febdb24033ac8e8b5b0989eb65" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 38963 } } peers { permanent_uuid: "413f356d0519462e94edbb0e204295eb" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39053 } } peers { permanent_uuid: "58d6eebbe3a34b44a4c1e5b30f046814" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 34987 } }
I20260809 19:25:48.060128 27219 raft_consensus.cc:374] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:48.060180 27219 raft_consensus.cc:729] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe8415febdb24033ac8e8b5b0989eb65, State: Initialized, Role: FOLLOWER
I20260809 19:25:48.060298 27219 consensus_queue.cc:260] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe8415febdb24033ac8e8b5b0989eb65" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 38963 } } peers { permanent_uuid: "413f356d0519462e94edbb0e204295eb" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39053 } } peers { permanent_uuid: "58d6eebbe3a34b44a4c1e5b30f046814" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 34987 } }
I20260809 19:25:48.060566 27219 ts_tablet_manager.cc:1428] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260809 19:25:48.060747 27205 heartbeater.cc:503] Master 127.21.81.254:42395 was elected leader, sending a full tablet report...
I20260809 19:25:48.061164 27223 raft_consensus.cc:348] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe8415febdb24033ac8e8b5b0989eb65" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 38963 } } peers { permanent_uuid: "413f356d0519462e94edbb0e204295eb" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39053 } } peers { permanent_uuid: "58d6eebbe3a34b44a4c1e5b30f046814" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 34987 } }
I20260809 19:25:48.061319 27223 raft_consensus.cc:374] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:48.061404 27223 raft_consensus.cc:729] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58d6eebbe3a34b44a4c1e5b30f046814, State: Initialized, Role: FOLLOWER
I20260809 19:25:48.061514 27223 consensus_queue.cc:260] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe8415febdb24033ac8e8b5b0989eb65" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 38963 } } peers { permanent_uuid: "413f356d0519462e94edbb0e204295eb" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39053 } } peers { permanent_uuid: "58d6eebbe3a34b44a4c1e5b30f046814" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 34987 } }
I20260809 19:25:48.061777 27223 ts_tablet_manager.cc:1428] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:25:48.061853 26932 heartbeater.cc:503] Master 127.21.81.254:42395 was elected leader, sending a full tablet report...
W20260809 19:25:48.164273 27070 tablet.cc:2369] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:25:48.282191 27206 tablet.cc:2369] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:25:48.282734 27227 raft_consensus.cc:482] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:25:48.282848 27227 raft_consensus.cc:504] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe8415febdb24033ac8e8b5b0989eb65" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 38963 } } peers { permanent_uuid: "413f356d0519462e94edbb0e204295eb" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39053 } } peers { permanent_uuid: "58d6eebbe3a34b44a4c1e5b30f046814" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 34987 } }
I20260809 19:25:48.283179 27227 leader_election.cc:288] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe8415febdb24033ac8e8b5b0989eb65 (127.21.81.195:38963), 413f356d0519462e94edbb0e204295eb (127.21.81.194:39053)
I20260809 19:25:48.284703 26934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.283411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:39053 (local address 127.21.81.193:46325)
0809 19:25:48.283486 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:48.283500 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:48.283563 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:48.283741 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:48.283742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:48.283746 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:48.284321 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:25:48.284322 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:48.284329 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:25:48.284340 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:48.284529 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:48.284530 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:48.284531 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:25:48.284583 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 19:25:48.284597 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 19:25:48.284795 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.283532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:46325 (local address 127.21.81.194:39053)
0809 19:25:48.283584 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:48.283586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:48.283598 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:25:48.283630 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:48.283632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:48.283638 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:48.283711 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:48.284428 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:48.284430 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:48.284430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:48.284435 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:48.284455 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:48.284465 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:48.284533 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:48.284703 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 19:25:48.284719 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:25:48.284914 27216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.283547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:59163 (local address 127.21.81.195:38963)
0809 19:25:48.283592 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:48.283595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:48.283759 (+ 164us) server_negotiation.cc:407] Connection header received
0809 19:25:48.283800 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:48.283801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:48.283808 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:48.283872 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:48.284590 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:48.284592 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:48.284592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:48.284597 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:48.284614 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:48.284618 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:48.284674 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:48.284800 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 19:25:48.284821 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:25:48.285100 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.283413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:38963 (local address 127.21.81.193:59163)
0809 19:25:48.283675 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:48.283685 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:48.283750 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:48.283886 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:48.283887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:48.283889 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:48.284500 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:25:48.284501 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:48.284515 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:25:48.284527 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:48.284686 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:48.284687 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:48.284688 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:25:48.285018 (+ 330us) client_negotiation.cc:240] Negotiation successful
0809 19:25:48.285033 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":86,"thread_start_us":111,"threads_started":1}
I20260809 19:25:48.285121 27020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a558c4904e0e4c02b33b73e93f4d245a" candidate_uuid: "58d6eebbe3a34b44a4c1e5b30f046814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413f356d0519462e94edbb0e204295eb" is_pre_election: true
I20260809 19:25:48.285310 27020 raft_consensus.cc:2454] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58d6eebbe3a34b44a4c1e5b30f046814 in term 0.
I20260809 19:25:48.285553 27158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a558c4904e0e4c02b33b73e93f4d245a" candidate_uuid: "58d6eebbe3a34b44a4c1e5b30f046814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe8415febdb24033ac8e8b5b0989eb65" is_pre_election: true
I20260809 19:25:48.285640 26816 leader_election.cc:302] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413f356d0519462e94edbb0e204295eb, 58d6eebbe3a34b44a4c1e5b30f046814; no voters:
I20260809 19:25:48.285764 27158 raft_consensus.cc:2454] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58d6eebbe3a34b44a4c1e5b30f046814 in term 0.
I20260809 19:25:48.285787 27227 raft_consensus.cc:2781] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:25:48.285854 27227 raft_consensus.cc:482] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:25:48.285890 27227 raft_consensus.cc:3037] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:48.286638 27227 raft_consensus.cc:504] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe8415febdb24033ac8e8b5b0989eb65" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 38963 } } peers { permanent_uuid: "413f356d0519462e94edbb0e204295eb" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39053 } } peers { permanent_uuid: "58d6eebbe3a34b44a4c1e5b30f046814" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 34987 } }
I20260809 19:25:48.286801 27227 leader_election.cc:288] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [CANDIDATE]: Term 1 election: Requested vote from peers fe8415febdb24033ac8e8b5b0989eb65 (127.21.81.195:38963), 413f356d0519462e94edbb0e204295eb (127.21.81.194:39053)
I20260809 19:25:48.286964 27158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a558c4904e0e4c02b33b73e93f4d245a" candidate_uuid: "58d6eebbe3a34b44a4c1e5b30f046814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe8415febdb24033ac8e8b5b0989eb65"
I20260809 19:25:48.287030 27158 raft_consensus.cc:3037] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:48.287775 27158 raft_consensus.cc:2454] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58d6eebbe3a34b44a4c1e5b30f046814 in term 1.
I20260809 19:25:48.287909 27020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a558c4904e0e4c02b33b73e93f4d245a" candidate_uuid: "58d6eebbe3a34b44a4c1e5b30f046814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413f356d0519462e94edbb0e204295eb"
I20260809 19:25:48.287961 26815 leader_election.cc:302] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58d6eebbe3a34b44a4c1e5b30f046814, fe8415febdb24033ac8e8b5b0989eb65; no voters:
I20260809 19:25:48.287997 27020 raft_consensus.cc:3037] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:48.288060 27227 raft_consensus.cc:2781] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:48.288218 27227 raft_consensus.cc:686] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [term 1 LEADER]: Becoming Leader. State: Replica: 58d6eebbe3a34b44a4c1e5b30f046814, State: Running, Role: LEADER
I20260809 19:25:48.288338 27227 consensus_queue.cc:237] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe8415febdb24033ac8e8b5b0989eb65" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 38963 } } peers { permanent_uuid: "413f356d0519462e94edbb0e204295eb" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39053 } } peers { permanent_uuid: "58d6eebbe3a34b44a4c1e5b30f046814" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 34987 } }
I20260809 19:25:48.289033 27020 raft_consensus.cc:2454] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58d6eebbe3a34b44a4c1e5b30f046814 in term 1.
I20260809 19:25:48.289323 26735 catalog_manager.cc:5495] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58d6eebbe3a34b44a4c1e5b30f046814 (127.21.81.193). New cstate: current_term: 1 leader_uuid: "58d6eebbe3a34b44a4c1e5b30f046814" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe8415febdb24033ac8e8b5b0989eb65" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 38963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413f356d0519462e94edbb0e204295eb" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58d6eebbe3a34b44a4c1e5b30f046814" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 34987 } health_report { overall_health: HEALTHY } } }
W20260809 19:25:48.301476 26933 tablet.cc:2369] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:25:48.333766 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.332700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54336 (local address 127.21.81.193:34987)
0809 19:25:48.332762 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:48.332764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:48.332774 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:25:48.332816 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:48.332817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:48.332830 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:48.332875 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:48.333491 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:48.333492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:48.333492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:48.333495 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:48.333506 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:48.333508 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:48.333548 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:48.333643 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 19:25:48.333657 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:25:48.335626 27020 raft_consensus.cc:1261] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb [term 1 FOLLOWER]: Refusing update from remote peer 58d6eebbe3a34b44a4c1e5b30f046814: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:48.335638 27158 raft_consensus.cc:1261] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65 [term 1 FOLLOWER]: Refusing update from remote peer 58d6eebbe3a34b44a4c1e5b30f046814: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:48.335995 27230 consensus_queue.cc:1035] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [LEADER]: Connected to new peer: Peer: permanent_uuid: "413f356d0519462e94edbb0e204295eb" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:48.336146 27227 consensus_queue.cc:1035] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe8415febdb24033ac8e8b5b0989eb65" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 38963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:48.339526 27234 mvcc.cc:205] Tried to move back new op lower bound from 7316699333979783168 to 7316699333789458432. Current Snapshot: MvccSnapshot[applied={T|T < 7316699333979783168 or (T in {7316699333979783168})}]
I20260809 19:25:48.340694 26734 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:48.340796 26734 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:48.341677 26734 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d83ad830355a4d76b9bf7a78c15355f2: Sending DeleteTablet for 3 replicas of tablet a558c4904e0e4c02b33b73e93f4d245a
I20260809 19:25:48.341993 26998 tablet_service.cc:1514] Processing DeleteTablet for tablet a558c4904e0e4c02b33b73e93f4d245a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:48 UTC) from {username='slave'} at 127.0.0.1:40402
I20260809 19:25:48.341979 26863 tablet_service.cc:1514] Processing DeleteTablet for tablet a558c4904e0e4c02b33b73e93f4d245a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:48 UTC) from {username='slave'} at 127.0.0.1:54324
I20260809 19:25:48.342114 27138 tablet_service.cc:1514] Processing DeleteTablet for tablet a558c4904e0e4c02b33b73e93f4d245a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:48 UTC) from {username='slave'} at 127.0.0.1:41578
I20260809 19:25:48.342217 27244 tablet_replica.cc:330] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb: stopping tablet replica
I20260809 19:25:48.342217 27137 tablet_service.cc:1514] Processing DeleteTablet for tablet a558c4904e0e4c02b33b73e93f4d245a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:48 UTC) from {username='slave'} at 127.0.0.1:41578
I20260809 19:25:48.342316 27244 raft_consensus.cc:2229] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:48.342408 27243 tablet_replica.cc:330] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814: stopping tablet replica
I20260809 19:25:48.342422 27244 raft_consensus.cc:2258] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:48.342499 27243 raft_consensus.cc:2229] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:25:48.342584 27243 raft_consensus.cc:2258] T a558c4904e0e4c02b33b73e93f4d245a P 58d6eebbe3a34b44a4c1e5b30f046814 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:48.342804 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 26799
I20260809 19:25:48.348592 27244 ts_tablet_manager.cc:1905] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:48.350256 27244 ts_tablet_manager.cc:1918] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:25:48.350330 27244 log.cc:1196] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/a558c4904e0e4c02b33b73e93f4d245a
I20260809 19:25:48.350577 27244 ts_tablet_manager.cc:1939] T a558c4904e0e4c02b33b73e93f4d245a P 413f356d0519462e94edbb0e204295eb: Deleting consensus metadata
I20260809 19:25:48.350773 27245 tablet_replica.cc:330] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65: stopping tablet replica
I20260809 19:25:48.350873 27245 raft_consensus.cc:2229] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:48.350963 27245 raft_consensus.cc:2258] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:48.351375 27245 ts_tablet_manager.cc:1905] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:48.352743 26719 catalog_manager.cc:4841] TS 413f356d0519462e94edbb0e204295eb (127.21.81.194:39053): tablet a558c4904e0e4c02b33b73e93f4d245a (table test-table [id=9ab229ac330d4ac68070cdce0687389f]) successfully deleted
I20260809 19:25:48.352921 27245 ts_tablet_manager.cc:1918] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:25:48.353003 27245 log.cc:1196] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/a558c4904e0e4c02b33b73e93f4d245a
I20260809 19:25:48.353269 27245 ts_tablet_manager.cc:1939] T a558c4904e0e4c02b33b73e93f4d245a P fe8415febdb24033ac8e8b5b0989eb65: Deleting consensus metadata
I20260809 19:25:48.353685 26718 catalog_manager.cc:4841] TS fe8415febdb24033ac8e8b5b0989eb65 (127.21.81.195:38963): tablet a558c4904e0e4c02b33b73e93f4d245a (table test-table [id=9ab229ac330d4ac68070cdce0687389f]) successfully deleted
W20260809 19:25:48.353806 26721 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:34987 (error 108)
W20260809 19:25:48.353948 26718 catalog_manager.cc:4808] TS fe8415febdb24033ac8e8b5b0989eb65 (127.21.81.195:38963): delete failed for tablet a558c4904e0e4c02b33b73e93f4d245a because the tablet was not found. No further retry: Not found: Tablet not found: a558c4904e0e4c02b33b73e93f4d245a
I20260809 19:25:48.354110 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 26935
I20260809 19:25:48.355120 27214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.354258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:34987 (local address 127.0.0.1:54346)
0809 19:25:48.354995 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:48.355079 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":712}
W20260809 19:25:48.355273 26721 catalog_manager.cc:4584] TS 58d6eebbe3a34b44a4c1e5b30f046814 (127.21.81.193:34987): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a558c4904e0e4c02b33b73e93f4d245a: Network error: Client connection negotiation failed: client connection to 127.21.81.193:34987: connect: Connection refused (error 111)
I20260809 19:25:48.361792 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 27072
I20260809 19:25:48.367575 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 26703
2026-08-09T19:25:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (862 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 19:25:48.389784 21831 mini_oidc.cc:152] Starting JWKS server
I20260809 19:25:48.390951 21831 webserver.cc:460] Webserver started at https://127.0.0.1:40397/ using document root <none> and password file <none>
I20260809 19:25:48.391077 21831 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 19:25:48.391268 21831 webserver.cc:460] Webserver started at http://127.0.0.1:37869/ using document root <none> and password file <none>
2026-08-09T19:25:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:48Z Disabled control of system clock
I20260809 19:25:48.400063 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39573
--webserver_interface=127.21.81.254
--webserver_port=0
--builtin_ntp_servers=127.21.81.212:39429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:39573
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40397/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/testchainca.cert with env {}
W20260809 19:25:48.482488 27252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:48.482714 27252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:48.482738 27252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:48.483959 27252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:48.484004 27252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:48.484016 27252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:48.484055 27252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:48.484084 27252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 19:25:48.484102 27252 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 19:25:48.484123 27252 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40397/jwks.json
I20260809 19:25:48.485783 27252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:39429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:39573
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40397/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:48.486049 27252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:48.486292 27252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:48.491086 27258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:48.491103 27261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:48.491086 27259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:48.491418 27252 server_base.cc:1034] running on GCE node
I20260809 19:25:48.491677 27252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:48.492028 27252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:48.493175 27252 hybrid_clock.cc:648] HybridClock initialized: now 1786303548493162 us; error 30 us; skew 500 ppm
I20260809 19:25:48.494450 27252 webserver.cc:460] Webserver started at http://127.21.81.254:43021/ using document root <none> and password file <none>
I20260809 19:25:48.494709 27252 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:48.494755 27252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:48.494899 27252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:48.495991 27252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "aa277ff8f6a54b3abf1a20d5d091c451"
format_stamp: "Formatted at 2026-08-09 19:25:48 on dist-test-slave-njxd"
server_key: "5657c18bb9fdb188429daa1deedd7125"
server_key_iv: "94800a92b4b582e80ec75402835b8b8a"
server_key_version: "encryptionkey@0"
I20260809 19:25:48.496379 27252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "aa277ff8f6a54b3abf1a20d5d091c451"
format_stamp: "Formatted at 2026-08-09 19:25:48 on dist-test-slave-njxd"
server_key: "5657c18bb9fdb188429daa1deedd7125"
server_key_iv: "94800a92b4b582e80ec75402835b8b8a"
server_key_version: "encryptionkey@0"
I20260809 19:25:48.497736 27252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:25:48.498481 27268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:48.498651 27252 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:48.498742 27252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "aa277ff8f6a54b3abf1a20d5d091c451"
format_stamp: "Formatted at 2026-08-09 19:25:48 on dist-test-slave-njxd"
server_key: "5657c18bb9fdb188429daa1deedd7125"
server_key_iv: "94800a92b4b582e80ec75402835b8b8a"
server_key_version: "encryptionkey@0"
I20260809 19:25:48.498821 27252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:48.509509 27252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:48.509905 27252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:48.510066 27252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:48.514386 27252 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:39573
I20260809 19:25:48.514422 27322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:39573 every 8 connection(s)
I20260809 19:25:48.514827 27252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:48.515797 27323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:48.518250 27323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451: Bootstrap starting.
I20260809 19:25:48.518864 27323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:48.519170 27323 log.cc:824] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:48.519842 27323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451: No bootstrap required, opened a new log
I20260809 19:25:48.521399 27323 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa277ff8f6a54b3abf1a20d5d091c451" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39573 } }
I20260809 19:25:48.521535 27323 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:48.521616 27323 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa277ff8f6a54b3abf1a20d5d091c451, State: Initialized, Role: FOLLOWER
I20260809 19:25:48.521806 27323 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa277ff8f6a54b3abf1a20d5d091c451" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39573 } }
I20260809 19:25:48.521871 27323 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:48.521930 27323 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:48.521981 27323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:48.522507 27323 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa277ff8f6a54b3abf1a20d5d091c451" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39573 } }
I20260809 19:25:48.522624 27323 leader_election.cc:302] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa277ff8f6a54b3abf1a20d5d091c451; no voters:
I20260809 19:25:48.522818 27323 leader_election.cc:288] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:48.522872 27326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:48.523046 27326 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [term 1 LEADER]: Becoming Leader. State: Replica: aa277ff8f6a54b3abf1a20d5d091c451, State: Running, Role: LEADER
I20260809 19:25:48.523087 27323 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:48.523149 27326 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa277ff8f6a54b3abf1a20d5d091c451" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39573 } }
I20260809 19:25:48.523531 27329 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa277ff8f6a54b3abf1a20d5d091c451. Latest consensus state: current_term: 1 leader_uuid: "aa277ff8f6a54b3abf1a20d5d091c451" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa277ff8f6a54b3abf1a20d5d091c451" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39573 } } }
I20260809 19:25:48.523645 27329 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:48.523631 27328 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa277ff8f6a54b3abf1a20d5d091c451" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa277ff8f6a54b3abf1a20d5d091c451" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39573 } } }
I20260809 19:25:48.523936 27328 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:48.524109 27337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:48.524595 27337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:48.524888 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 27252
I20260809 19:25:48.524991 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:48.525179 21831 external_mini_cluster.cc:1428] Setting key 7c7deba193d79ba268b78037c4f75b0f
I20260809 19:25:48.526347 27337 catalog_manager.cc:1347] Generated new cluster ID: 0aa4cb9dd6004371b34e5867d0cd4e47
I20260809 19:25:48.526401 27337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:48.531769 27347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.525725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47038 (local address 127.21.81.254:39573)
0809 19:25:48.525935 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:48.525944 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:48.525976 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:25:48.526086 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:48.526093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:48.526159 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:48.526321 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:48.526666 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:48.528040 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:48.529189 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:48.529864 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:48.529977 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:48.530919 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:48.530923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:48.530923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:48.531139 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:48.531180 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:48.531194 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:48.531267 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:48.531479 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 19:25:48.531560 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260809 19:25:48.534132 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:25:48.538244 27337 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:48.538724 27337 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:48.544888 27337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa277ff8f6a54b3abf1a20d5d091c451: Generated new TSK 0
I20260809 19:25:48.545080 27337 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 19:25:48.568660 27355 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 19:25:48.568806 27355 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 19:25:48.569041 27347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.553598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47050 (local address 127.21.81.254:39573)
0809 19:25:48.553650 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:48.553652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:48.553806 (+ 154us) server_negotiation.cc:407] Connection header received
0809 19:25:48.553901 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:48.553902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:48.553913 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:48.553970 (+ 57us) server_negotiation.cc:226] Negotiated authn=JWT
0809 19:25:48.554126 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:48.554604 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:48.555748 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:48.555923 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:48.555975 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:48.556196 (+ 221us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 19:25:48.568789 (+ 12593us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40397/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 19:25:48.568963 (+ 174us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47050: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40397/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":25}
W20260809 19:25:48.569100 27347 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47050: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40397/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 19:25:48.569434 27354 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:48.553519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39573 (local address 127.0.0.1:47050)
0809 19:25:48.553724 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:48.553738 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:48.553811 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:48.553997 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:48.553998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:48.554000 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 19:25:48.554062 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:48.554064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:48.554697 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:48.554698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:48.555663 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:48.555665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:48.556005 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:48.556007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:48.556071 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:48.556093 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 19:25:48.569014 (+ 12921us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40397/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 19:25:48.569151 (+ 137us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.81.254:39573: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40397/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.21.81.254:39573: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40397/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 19:25:48.569948 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 27252
2026-08-09T19:25:48Z chronyd exiting
I20260809 19:25:48.609751 21831 test_util.cc:175] -----------------------------------------------
I20260809 19:25:48.609830 21831 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303525464116-21831-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (221 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 19:25:48.610585 21831 mini_oidc.cc:152] Starting JWKS server
I20260809 19:25:48.611238 21831 webserver.cc:460] Webserver started at https://127.0.0.1:45289/ using document root <none> and password file <none>
I20260809 19:25:48.611279 21831 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 19:25:48.611388 21831 webserver.cc:460] Webserver started at http://127.0.0.1:42553/ using document root <none> and password file <none>
2026-08-09T19:25:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:48Z Disabled control of system clock
I20260809 19:25:48.619930 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:42219
--webserver_interface=127.21.81.254
--webserver_port=0
--builtin_ntp_servers=127.21.81.212:37721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:42219
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45289/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/testchainca.cert with env {}
W20260809 19:25:48.703575 27361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:48.703797 27361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:48.703820 27361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:48.704938 27361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:48.704979 27361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:48.704993 27361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:48.705005 27361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:48.705017 27361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 19:25:48.705029 27361 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 19:25:48.705041 27361 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45289/jwks.json
I20260809 19:25:48.706634 27361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:42219
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:42219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45289/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:48.706858 27361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:48.707029 27361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:48.711935 27366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:48.711935 27369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:48.711943 27367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:48.712220 27361 server_base.cc:1034] running on GCE node
I20260809 19:25:48.712469 27361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:48.712723 27361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:48.713881 27361 hybrid_clock.cc:648] HybridClock initialized: now 1786303548713864 us; error 29 us; skew 500 ppm
I20260809 19:25:48.715210 27361 webserver.cc:460] Webserver started at http://127.21.81.254:35569/ using document root <none> and password file <none>
I20260809 19:25:48.715467 27361 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:48.715515 27361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:48.715694 27361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:48.716791 27361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "646bff6e7eb6457bab3eb495a3e28ebc"
format_stamp: "Formatted at 2026-08-09 19:25:48 on dist-test-slave-njxd"
server_key: "14666d5dddc6cb21e621decd83d0b8b7"
server_key_iv: "41e06b65f9f11cf3a1f873a4e7cd7138"
server_key_version: "encryptionkey@0"
I20260809 19:25:48.717171 27361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "646bff6e7eb6457bab3eb495a3e28ebc"
format_stamp: "Formatted at 2026-08-09 19:25:48 on dist-test-slave-njxd"
server_key: "14666d5dddc6cb21e621decd83d0b8b7"
server_key_iv: "41e06b65f9f11cf3a1f873a4e7cd7138"
server_key_version: "encryptionkey@0"
I20260809 19:25:48.718561 27361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:25:48.719313 27375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:48.719485 27361 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:48.719596 27361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "646bff6e7eb6457bab3eb495a3e28ebc"
format_stamp: "Formatted at 2026-08-09 19:25:48 on dist-test-slave-njxd"
server_key: "14666d5dddc6cb21e621decd83d0b8b7"
server_key_iv: "41e06b65f9f11cf3a1f873a4e7cd7138"
server_key_version: "encryptionkey@0"
I20260809 19:25:48.719692 27361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:48.740507 27361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:48.740906 27361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:48.741067 27361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:48.745422 27361 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:42219
I20260809 19:25:48.745460 27429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:42219 every 8 connection(s)
I20260809 19:25:48.745888 27361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:48.746618 27430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:48.749536 27430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc: Bootstrap starting.
I20260809 19:25:48.750196 27430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:48.750451 27430 log.cc:824] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:48.751137 27430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc: No bootstrap required, opened a new log
I20260809 19:25:48.752710 27430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646bff6e7eb6457bab3eb495a3e28ebc" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42219 } }
I20260809 19:25:48.752852 27430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:48.752876 27430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 646bff6e7eb6457bab3eb495a3e28ebc, State: Initialized, Role: FOLLOWER
I20260809 19:25:48.753023 27430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646bff6e7eb6457bab3eb495a3e28ebc" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42219 } }
I20260809 19:25:48.753108 27430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:48.753132 27430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:48.753161 27430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:48.753758 27430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646bff6e7eb6457bab3eb495a3e28ebc" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42219 } }
I20260809 19:25:48.753847 27430 leader_election.cc:302] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 646bff6e7eb6457bab3eb495a3e28ebc; no voters:
I20260809 19:25:48.753976 27430 leader_election.cc:288] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:48.754065 27433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:48.754243 27433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [term 1 LEADER]: Becoming Leader. State: Replica: 646bff6e7eb6457bab3eb495a3e28ebc, State: Running, Role: LEADER
I20260809 19:25:48.754284 27430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:48.754350 27433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646bff6e7eb6457bab3eb495a3e28ebc" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42219 } }
I20260809 19:25:48.754802 27434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "646bff6e7eb6457bab3eb495a3e28ebc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646bff6e7eb6457bab3eb495a3e28ebc" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42219 } } }
I20260809 19:25:48.754911 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 27361
I20260809 19:25:48.755029 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:48.755219 21831 external_mini_cluster.cc:1428] Setting key 3e4c4777f7ece10bcc0bf4e7a9fa929d
I20260809 19:25:48.755259 27435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 646bff6e7eb6457bab3eb495a3e28ebc. Latest consensus state: current_term: 1 leader_uuid: "646bff6e7eb6457bab3eb495a3e28ebc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646bff6e7eb6457bab3eb495a3e28ebc" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42219 } } }
I20260809 19:25:48.755348 27434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:48.755662 27435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc [sys.catalog]: This master's current role is: LEADER
W20260809 19:25:48.756722 27452 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:25:48.756781 27452 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:25:48.756839 27447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:48.756992 27447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:48.759086 27447 catalog_manager.cc:1347] Generated new cluster ID: fed1562851834ee5962b1ccbe1c9afc8
I20260809 19:25:48.759150 27447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:48.761229 27449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.755902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53118 (local address 127.21.81.254:42219)
0809 19:25:48.756083 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:48.756093 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:48.756130 (+ 37us) server_negotiation.cc:407] Connection header received
0809 19:25:48.756254 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:48.756261 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:48.756333 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:48.756476 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:48.756745 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:48.758038 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:48.759029 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:48.759635 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:48.759754 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:48.760516 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:48.760520 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:48.760520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:48.760740 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:48.760781 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:48.760795 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:48.760855 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:48.760951 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 19:25:48.761025 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":84,"threads_started":1}
I20260809 19:25:48.762079 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:25:48.782042 27447 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:48.782804 27447 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:48.794198 27447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 646bff6e7eb6457bab3eb495a3e28ebc: Generated new TSK 0
I20260809 19:25:48.794430 27447 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 19:25:48.819047 27461 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 19:25:48.819149 27461 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 19:25:48.819525 27449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.805305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53122 (local address 127.21.81.254:42219)
0809 19:25:48.805360 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:48.805363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:48.805388 (+ 25us) server_negotiation.cc:407] Connection header received
0809 19:25:48.805479 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:48.805480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:48.805492 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:48.805579 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0809 19:25:48.805769 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:48.806278 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:48.807109 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:48.807321 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:48.807399 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:48.807599 (+ 200us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 19:25:48.819273 (+ 11674us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45289/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 19:25:48.819446 (+ 173us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53122: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45289/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 19:25:48.819589 27449 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53122: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45289/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 19:25:48.819600 27460 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:48.805155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42219 (local address 127.0.0.1:53122)
0809 19:25:48.805321 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:48.805334 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:48.805410 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:48.805609 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:48.805610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:48.805618 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 19:25:48.805685 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:48.805689 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:48.806386 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:48.806387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:48.806998 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:48.806999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:48.807400 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:48.807401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:48.807466 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:48.807471 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 19:25:48.819391 (+ 11920us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45289/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 19:25:48.819498 (+ 107us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.81.254:42219: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45289/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":112,"threads_started":1}
I20260809 19:25:48.819904 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 27361
2026-08-09T19:25:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (249 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 19:25:48.860194 21831 mini_oidc.cc:152] Starting JWKS server
I20260809 19:25:48.860896 21831 webserver.cc:460] Webserver started at https://127.0.0.1:43833/ using document root <none> and password file <none>
I20260809 19:25:48.860970 21831 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 19:25:48.861148 21831 webserver.cc:460] Webserver started at http://127.0.0.1:38721/ using document root <none> and password file <none>
2026-08-09T19:25:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:48Z Disabled control of system clock
I20260809 19:25:48.869843 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:45865
--webserver_interface=127.21.81.254
--webserver_port=0
--builtin_ntp_servers=127.21.81.212:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:45865
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43833/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 19:25:48.949721 27467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:48.949919 27467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:48.949940 27467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:48.951000 27467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:48.951032 27467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:48.951045 27467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:48.951056 27467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:48.951067 27467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 19:25:48.951079 27467 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 19:25:48.951090 27467 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43833/jwks.json
I20260809 19:25:48.952340 27467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:45865
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:45865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43833/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:48.952521 27467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:48.952740 27467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:25:48.957468 27467 server_base.cc:1034] running on GCE node
W20260809 19:25:48.957427 27472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:48.957438 27475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:48.957486 27473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:48.957890 27467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:48.958230 27467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:48.959398 27467 hybrid_clock.cc:648] HybridClock initialized: now 1786303548959392 us; error 34 us; skew 500 ppm
I20260809 19:25:48.960706 27467 webserver.cc:460] Webserver started at http://127.21.81.254:35831/ using document root <none> and password file <none>
I20260809 19:25:48.960978 27467 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:48.961050 27467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:48.961217 27467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:48.962294 27467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "48f90729a810448a96ceaa0968fd5820"
format_stamp: "Formatted at 2026-08-09 19:25:48 on dist-test-slave-njxd"
server_key: "3e0c5cf3556d6926b1777cb78d0b5d7d"
server_key_iv: "deecec58782419cb7774f8bcd9428eb6"
server_key_version: "encryptionkey@0"
I20260809 19:25:48.962648 27467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "48f90729a810448a96ceaa0968fd5820"
format_stamp: "Formatted at 2026-08-09 19:25:48 on dist-test-slave-njxd"
server_key: "3e0c5cf3556d6926b1777cb78d0b5d7d"
server_key_iv: "deecec58782419cb7774f8bcd9428eb6"
server_key_version: "encryptionkey@0"
I20260809 19:25:48.964000 27467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:25:48.964756 27482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:48.964917 27467 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:48.965018 27467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "48f90729a810448a96ceaa0968fd5820"
format_stamp: "Formatted at 2026-08-09 19:25:48 on dist-test-slave-njxd"
server_key: "3e0c5cf3556d6926b1777cb78d0b5d7d"
server_key_iv: "deecec58782419cb7774f8bcd9428eb6"
server_key_version: "encryptionkey@0"
I20260809 19:25:48.965086 27467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:48.975682 27467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:48.976086 27467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:48.976251 27467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:48.980402 27467 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:45865
I20260809 19:25:48.980434 27536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:45865 every 8 connection(s)
I20260809 19:25:48.980757 27467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:48.981412 27537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:48.983891 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 27467
I20260809 19:25:48.984006 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:48.984164 21831 external_mini_cluster.cc:1428] Setting key 142676d97f47430c9b5d569da7217757
I20260809 19:25:48.984254 27537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820: Bootstrap starting.
I20260809 19:25:48.985008 27537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:48.985350 27537 log.cc:824] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:48.986272 27537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820: No bootstrap required, opened a new log
I20260809 19:25:48.988525 27537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48f90729a810448a96ceaa0968fd5820" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45865 } }
I20260809 19:25:48.988689 27537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:48.988723 27537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48f90729a810448a96ceaa0968fd5820, State: Initialized, Role: FOLLOWER
I20260809 19:25:48.988849 27537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48f90729a810448a96ceaa0968fd5820" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45865 } }
I20260809 19:25:48.988940 27537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:48.988984 27537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:48.989032 27537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:48.989648 27537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48f90729a810448a96ceaa0968fd5820" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45865 } }
I20260809 19:25:48.989760 27537 leader_election.cc:302] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48f90729a810448a96ceaa0968fd5820; no voters:
I20260809 19:25:48.989913 27537 leader_election.cc:288] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:48.989948 27543 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:48.990124 27543 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [term 1 LEADER]: Becoming Leader. State: Replica: 48f90729a810448a96ceaa0968fd5820, State: Running, Role: LEADER
I20260809 19:25:48.990177 27537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:48.990221 27543 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48f90729a810448a96ceaa0968fd5820" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45865 } }
I20260809 19:25:48.990540 27545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48f90729a810448a96ceaa0968fd5820. Latest consensus state: current_term: 1 leader_uuid: "48f90729a810448a96ceaa0968fd5820" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48f90729a810448a96ceaa0968fd5820" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45865 } } }
I20260809 19:25:48.990554 27544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48f90729a810448a96ceaa0968fd5820" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48f90729a810448a96ceaa0968fd5820" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45865 } } }
I20260809 19:25:48.990659 27545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:48.990665 27544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:48.990896 27540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:48.984656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.21.81.254:45865)
0809 19:25:48.984879 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:48.984887 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:48.984920 (+ 33us) server_negotiation.cc:407] Connection header received
0809 19:25:48.985553 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:48.985562 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:48.985674 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:48.985792 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:48.986058 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:48.987449 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:48.988256 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:48.988870 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:48.989017 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:48.989753 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:48.989756 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:48.989757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:48.989972 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:48.990019 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:48.990035 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:48.990106 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:48.990428 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 19:25:48.990514 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
I20260809 19:25:48.990962 27550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:48.992075 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:25:48.992077 27550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:48.993623 27550 catalog_manager.cc:1347] Generated new cluster ID: e51b2ecbcf0d45b1906142f53a818a10
I20260809 19:25:48.993695 27550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:49.013281 27550 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:49.013890 27550 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:49.016938 27550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48f90729a810448a96ceaa0968fd5820: Generated new TSK 0
I20260809 19:25:49.017132 27550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:25:49.026621 27540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:49.022882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42442 (local address 127.21.81.254:45865)
0809 19:25:49.022951 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:49.022953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:49.023017 (+ 64us) server_negotiation.cc:407] Connection header received
0809 19:25:49.023082 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:49.023083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:49.023095 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:49.023172 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0809 19:25:49.023292 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:49.024094 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:49.024865 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:49.025177 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:49.025248 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:49.025416 (+ 168us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 19:25:49.026407 (+ 991us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43833/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 19:25:49.026539 (+ 132us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42442: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43833/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 19:25:49.026678 27540 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42442: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43833/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 19:25:49.028561 27570 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:49.022794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:45865 (local address 127.0.0.1:42442)
0809 19:25:49.022954 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:49.022964 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:49.023028 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:49.023177 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:49.023177 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:49.023179 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 19:25:49.023229 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:49.023231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:49.024181 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:49.024182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:49.024786 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:49.024788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:49.025269 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:49.025270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:49.025315 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:49.025320 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 19:25:49.026512 (+ 1192us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43833/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 19:25:49.028467 (+ 1955us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.81.254:45865: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43833/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":1749,"thread_start_us":109,"threads_started":1}
I20260809 19:25:49.029006 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 27467
E20260809 19:25:49.038771 27571 webserver.cc:533] Webserver: sslize failed:
E20260809 19:25:49.038858 27571 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T19:25:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (209 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T19:25:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:49Z Disabled control of system clock
I20260809 19:25:49.080272 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:34967
--webserver_interface=127.21.81.254
--webserver_port=0
--builtin_ntp_servers=127.21.81.212:46661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:34967
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/testchainca.cert with env {}
W20260809 19:25:49.159566 27575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:49.159787 27575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:49.159811 27575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:49.160945 27575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:49.160979 27575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:49.160992 27575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:49.161005 27575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:49.161016 27575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 19:25:49.161028 27575 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 19:25:49.161039 27575 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 19:25:49.162520 27575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:34967
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:34967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:49.162741 27575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:49.162910 27575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:49.167654 27581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:49.167654 27580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:49.167655 27583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:49.167999 27575 server_base.cc:1034] running on GCE node
I20260809 19:25:49.168241 27575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:49.168519 27575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:49.169651 27575 hybrid_clock.cc:648] HybridClock initialized: now 1786303549169626 us; error 39 us; skew 500 ppm
I20260809 19:25:49.170845 27575 webserver.cc:460] Webserver started at http://127.21.81.254:42007/ using document root <none> and password file <none>
I20260809 19:25:49.171090 27575 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:49.171134 27575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:49.171299 27575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:49.172331 27575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "feb4c5453bbc4b57b36789a0c2815faa"
format_stamp: "Formatted at 2026-08-09 19:25:49 on dist-test-slave-njxd"
server_key: "7f936c76a11edc18de19a935d5c01511"
server_key_iv: "a4bc90675e2ade714c4f9ddc470f32b2"
server_key_version: "encryptionkey@0"
I20260809 19:25:49.172713 27575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "feb4c5453bbc4b57b36789a0c2815faa"
format_stamp: "Formatted at 2026-08-09 19:25:49 on dist-test-slave-njxd"
server_key: "7f936c76a11edc18de19a935d5c01511"
server_key_iv: "a4bc90675e2ade714c4f9ddc470f32b2"
server_key_version: "encryptionkey@0"
I20260809 19:25:49.174049 27575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:25:49.174803 27589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:49.174983 27575 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:49.175076 27575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "feb4c5453bbc4b57b36789a0c2815faa"
format_stamp: "Formatted at 2026-08-09 19:25:49 on dist-test-slave-njxd"
server_key: "7f936c76a11edc18de19a935d5c01511"
server_key_iv: "a4bc90675e2ade714c4f9ddc470f32b2"
server_key_version: "encryptionkey@0"
I20260809 19:25:49.175148 27575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:49.193084 27575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:49.193445 27575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:49.193655 27575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:49.198135 27575 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:34967
I20260809 19:25:49.198236 27645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:34967 every 8 connection(s)
I20260809 19:25:49.198612 27575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:49.199832 27646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:49.202536 27646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa: Bootstrap starting.
I20260809 19:25:49.203080 27646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:49.203404 27646 log.cc:824] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:49.204139 27646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa: No bootstrap required, opened a new log
I20260809 19:25:49.204625 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 27575
I20260809 19:25:49.204751 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:49.204919 21831 external_mini_cluster.cc:1428] Setting key 55b9465c8b34f632f433831fffea3f3b
I20260809 19:25:49.206277 27646 raft_consensus.cc:348] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb4c5453bbc4b57b36789a0c2815faa" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34967 } }
I20260809 19:25:49.206439 27646 raft_consensus.cc:374] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:49.206478 27646 raft_consensus.cc:729] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feb4c5453bbc4b57b36789a0c2815faa, State: Initialized, Role: FOLLOWER
I20260809 19:25:49.206611 27646 consensus_queue.cc:260] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb4c5453bbc4b57b36789a0c2815faa" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34967 } }
I20260809 19:25:49.206712 27646 raft_consensus.cc:388] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:49.206766 27646 raft_consensus.cc:482] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:49.206816 27646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:49.207369 27646 raft_consensus.cc:504] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb4c5453bbc4b57b36789a0c2815faa" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34967 } }
I20260809 19:25:49.207456 27646 leader_election.cc:302] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: feb4c5453bbc4b57b36789a0c2815faa; no voters:
I20260809 19:25:49.207630 27646 leader_election.cc:288] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:49.207667 27651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:49.207789 27651 raft_consensus.cc:686] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [term 1 LEADER]: Becoming Leader. State: Replica: feb4c5453bbc4b57b36789a0c2815faa, State: Running, Role: LEADER
I20260809 19:25:49.207891 27646 sys_catalog.cc:564] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:49.207868 27651 consensus_queue.cc:237] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb4c5453bbc4b57b36789a0c2815faa" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34967 } }
I20260809 19:25:49.208225 27651 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [sys.catalog]: SysCatalogTable state changed. Reason: New leader feb4c5453bbc4b57b36789a0c2815faa. Latest consensus state: current_term: 1 leader_uuid: "feb4c5453bbc4b57b36789a0c2815faa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb4c5453bbc4b57b36789a0c2815faa" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34967 } } }
I20260809 19:25:49.208295 27651 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:49.208474 27653 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "feb4c5453bbc4b57b36789a0c2815faa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb4c5453bbc4b57b36789a0c2815faa" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34967 } } }
I20260809 19:25:49.208552 27653 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:49.208717 27656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:49.209254 27656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:49.210933 27656 catalog_manager.cc:1347] Generated new cluster ID: 2df85480ab7f4a58acdd245452bf3b26
I20260809 19:25:49.210990 27656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:49.211480 27650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:49.205415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51178 (local address 127.21.81.254:34967)
0809 19:25:49.205672 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:49.205681 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:49.205718 (+ 37us) server_negotiation.cc:407] Connection header received
0809 19:25:49.206259 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:49.206266 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:49.206321 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:49.206425 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:49.206691 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:49.208211 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:49.209344 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:49.209930 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:49.210043 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:49.210715 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:49.210718 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:49.210719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:49.210933 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:49.210974 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:49.210988 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:49.211081 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:49.211184 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 19:25:49.211279 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":81,"threads_started":1}
I20260809 19:25:49.212280 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:25:49.218596 27656 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:49.219053 27656 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:49.226385 27656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P feb4c5453bbc4b57b36789a0c2815faa: Generated new TSK 0
I20260809 19:25:49.226567 27656 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:25:49.700543 27650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:49.235031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51190 (local address 127.21.81.254:34967)
0809 19:25:49.235092 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:49.235093 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:49.235212 (+ 119us) server_negotiation.cc:407] Connection header received
0809 19:25:49.235286 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:49.235287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:49.235297 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:49.235355 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0809 19:25:49.235515 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:49.235968 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:49.237125 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:49.237297 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:49.237341 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:49.237530 (+ 189us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 19:25:49.700307 (+462777us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 19:25:49.700481 (+ 174us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51190: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 19:25:49.700647 27650 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51190: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 19:25:49.700755 27675 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:49.234955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:34967 (local address 127.0.0.1:51190)
0809 19:25:49.235136 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:49.235149 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:49.235218 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:49.235381 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:49.235382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:49.235384 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 19:25:49.235449 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:49.235451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:49.236080 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:49.236081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:49.237039 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:49.237042 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:49.237375 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:49.237376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:49.237437 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:49.237442 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 19:25:49.700461 (+463019us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 19:25:49.700595 (+ 134us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.81.254:34967: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
I20260809 19:25:49.701175 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 27575
2026-08-09T19:25:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (652 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (93 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (87 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:49 dist-test-slave-njxd krb5kdc[27689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:49 dist-test-slave-njxd krb5kdc[27689](info): set up 2 sockets
Aug 09 19:25:49 dist-test-slave-njxd krb5kdc[27689](info): commencing operation
krb5kdc: starting...
W20260809 19:25:50.957263 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:25:51 dist-test-slave-njxd krb5kdc[27689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:25:51.078480 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:43881
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:43881
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:51.161976 27702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:51.162194 27702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:51.162215 27702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:51.163429 27702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:51.163475 27702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:51.163489 27702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:51.163502 27702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:51.163556 27702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:51.165091 27702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:43881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:43881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:51.165364 27702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:51.165656 27702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:51.171005 27707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:51.171013 27702 server_base.cc:1034] running on GCE node
W20260809 19:25:51.170984 27710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:51.170974 27708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:51.171551 27702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:51.171808 27702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:51.172993 27702 hybrid_clock.cc:648] HybridClock initialized: now 1786303551172981 us; error 49 us; skew 500 ppm
Aug 09 19:25:51 dist-test-slave-njxd krb5kdc[27689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303551, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:51.175670 27702 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:25:51.176168 27702 webserver.cc:460] Webserver started at http://127.21.81.254:42551/ using document root <none> and password file <none>
I20260809 19:25:51.176411 27702 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:51.176457 27702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:51.176548 27702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:51.178005 27702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "20ef94354b04406895746a40edf607d7"
format_stamp: "Formatted at 2026-08-09 19:25:51 on dist-test-slave-njxd"
server_key: "8fdd1e107f93aec0b1602ae75b9010b7"
server_key_iv: "8402fa8ba42e0443f918083aded56dcb"
server_key_version: "encryptionkey@0"
I20260809 19:25:51.178423 27702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "20ef94354b04406895746a40edf607d7"
format_stamp: "Formatted at 2026-08-09 19:25:51 on dist-test-slave-njxd"
server_key: "8fdd1e107f93aec0b1602ae75b9010b7"
server_key_iv: "8402fa8ba42e0443f918083aded56dcb"
server_key_version: "encryptionkey@0"
I20260809 19:25:51.180119 27702 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 19:25:51.181044 27719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:51.181253 27702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:51.181365 27702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "20ef94354b04406895746a40edf607d7"
format_stamp: "Formatted at 2026-08-09 19:25:51 on dist-test-slave-njxd"
server_key: "8fdd1e107f93aec0b1602ae75b9010b7"
server_key_iv: "8402fa8ba42e0443f918083aded56dcb"
server_key_version: "encryptionkey@0"
I20260809 19:25:51.181437 27702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:51.202400 27702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:51.204746 27702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:51.204952 27702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:51.209895 27702 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:43881
I20260809 19:25:51.209928 27773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:43881 every 8 connection(s)
I20260809 19:25:51.210282 27702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:51.211293 27774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:51.213737 27774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7: Bootstrap starting.
I20260809 19:25:51.213963 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 27702
I20260809 19:25:51.214072 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:51.214277 21831 external_mini_cluster.cc:1428] Setting key a5f7343a55b984ea9b4a00cd71ba3a9d
I20260809 19:25:51.214538 27774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:51.214938 27774 log.cc:824] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:51.215728 27774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7: No bootstrap required, opened a new log
I20260809 19:25:51.217763 27774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ef94354b04406895746a40edf607d7" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43881 } }
I20260809 19:25:51.217896 27774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:51.217919 27774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ef94354b04406895746a40edf607d7, State: Initialized, Role: FOLLOWER
I20260809 19:25:51.218056 27774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ef94354b04406895746a40edf607d7" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43881 } }
I20260809 19:25:51.218130 27774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:51.218152 27774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:51.218222 27774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 19:25:51 dist-test-slave-njxd krb5kdc[27689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:51.224445 27777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:51.214744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.21.81.254:43881)
0809 19:25:51.214963 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:51.214974 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:51.215011 (+ 37us) server_negotiation.cc:407] Connection header received
0809 19:25:51.215440 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:51.215446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:51.215492 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:51.215658 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:51.216338 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:51.217226 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:51.218275 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:51.218734 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:51.218866 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:51.221332 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:51.221336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:51.221336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:51.221347 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:51.223068 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:51.223391 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:51.223393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:51.223393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:51.223522 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:51.223703 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:51.223703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:51.223704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:51.223945 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:51.224043 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:51.224178 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 19:25:51.224286 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":105,"threads_started":1}
I20260809 19:25:51.277666 27774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ef94354b04406895746a40edf607d7" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43881 } }
I20260809 19:25:51.277886 27774 leader_election.cc:302] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20ef94354b04406895746a40edf607d7; no voters:
I20260809 19:25:51.278152 27774 leader_election.cc:288] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:51.278252 27779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:51.278476 27774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:51.278507 27779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [term 1 LEADER]: Becoming Leader. State: Replica: 20ef94354b04406895746a40edf607d7, State: Running, Role: LEADER
I20260809 19:25:51.278671 27779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ef94354b04406895746a40edf607d7" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43881 } }
I20260809 19:25:51.279098 27781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20ef94354b04406895746a40edf607d7. Latest consensus state: current_term: 1 leader_uuid: "20ef94354b04406895746a40edf607d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ef94354b04406895746a40edf607d7" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43881 } } }
I20260809 19:25:51.279130 27780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20ef94354b04406895746a40edf607d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ef94354b04406895746a40edf607d7" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43881 } } }
I20260809 19:25:51.279378 27780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:51.279276 27781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:51.280144 27784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:51.280666 27784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:51.282859 27784 catalog_manager.cc:1347] Generated new cluster ID: 9e2ec68a4a644b43a2eb903822602679
I20260809 19:25:51.282915 27784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:51.306391 27784 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:51.307190 27784 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:51.312851 27784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20ef94354b04406895746a40edf607d7: Generated new TSK 0
I20260809 19:25:51.313050 27784 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
W20260809 19:25:52.108539 21831 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.81.193: real 0.788s user 0.000s sys 0.004s
I20260809 19:25:52.108937 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43881
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:52.193995 27801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:52.194191 27801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:52.194216 27801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:52.195382 27801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:52.195425 27801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:52.195482 27801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:25:52.196983 27801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:52.197239 27801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:52.197474 27801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:52.202390 27809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:52.202406 27806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:52.202451 27807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:52.202674 27801 server_base.cc:1034] running on GCE node
I20260809 19:25:52.202900 27801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:52.203181 27801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:52.204350 27801 hybrid_clock.cc:648] HybridClock initialized: now 1786303552204331 us; error 31 us; skew 500 ppm
Aug 09 19:25:52 dist-test-slave-njxd krb5kdc[27689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303552, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:52.213392 27801 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:25:52.213992 27801 webserver.cc:460] Webserver started at http://127.21.81.193:45325/ using document root <none> and password file <none>
I20260809 19:25:52.214236 27801 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:52.214308 27801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:52.214478 27801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:52.215543 27801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "a252ea660bb34ed78174c4bb298d83b0"
format_stamp: "Formatted at 2026-08-09 19:25:52 on dist-test-slave-njxd"
server_key: "5e79ab7e11115101980942e89173d621"
server_key_iv: "98477ccb87016501a335c5e2ab35bc23"
server_key_version: "encryptionkey@0"
I20260809 19:25:52.215919 27801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "a252ea660bb34ed78174c4bb298d83b0"
format_stamp: "Formatted at 2026-08-09 19:25:52 on dist-test-slave-njxd"
server_key: "5e79ab7e11115101980942e89173d621"
server_key_iv: "98477ccb87016501a335c5e2ab35bc23"
server_key_version: "encryptionkey@0"
I20260809 19:25:52.217259 27801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:25:52.218111 27816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:52.218290 27801 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:52.218394 27801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "a252ea660bb34ed78174c4bb298d83b0"
format_stamp: "Formatted at 2026-08-09 19:25:52 on dist-test-slave-njxd"
server_key: "5e79ab7e11115101980942e89173d621"
server_key_iv: "98477ccb87016501a335c5e2ab35bc23"
server_key_version: "encryptionkey@0"
I20260809 19:25:52.218479 27801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:52.242580 27801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:52.244598 27801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:52.244776 27801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:52.245034 27801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:52.245371 27801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:52.245428 27801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:52.245469 27801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:52.245512 27801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:52.251710 27801 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:35995
I20260809 19:25:52.251773 27937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:35995 every 8 connection(s)
I20260809 19:25:52.252076 27801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:25:52.254671 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 27801
I20260809 19:25:52.254801 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:25:52.254998 21831 external_mini_cluster.cc:1428] Setting key 745381543b3b7b2bb22368c2bb59fc0b
Aug 09 19:25:52 dist-test-slave-njxd krb5kdc[27689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303552, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:52.262707 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:52.252817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:46705 (local address 127.21.81.254:43881)
0809 19:25:52.252957 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:52.252959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:52.253138 (+ 179us) server_negotiation.cc:407] Connection header received
0809 19:25:52.254232 (+ 1094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:52.254234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:52.254250 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:52.254341 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:52.255087 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:52.255735 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:52.256747 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:52.256962 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:52.257052 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:52.259561 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:52.259563 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:52.259563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:52.259572 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:52.261184 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:52.261668 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:52.261669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:52.261670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:52.261740 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:52.262011 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:52.262012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:52.262013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:52.262137 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:52.262211 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:52.262503 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 19:25:52.262597 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260809 19:25:52.262880 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:52.252800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:43881 (local address 127.21.81.193:46705)
0809 19:25:52.252989 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:52.253040 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:52.254101 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:52.254419 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:52.254423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:52.254719 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:52.254991 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:52.254997 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:52.255829 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:52.255830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:52.256653 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:52.256655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:52.257066 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:52.257067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:52.257216 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:52.257785 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:52.257794 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:52.259455 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:52.261293 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:52.261302 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:52.261304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:52.261544 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:52.261817 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:52.261817 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:52.261818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:52.261936 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:52.262215 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:52.262223 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:52.262366 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 19:25:52.262466 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 19:25:52.262636 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260809 19:25:52.263420 27938 heartbeater.cc:347] Connected to a master server at 127.21.81.254:43881
I20260809 19:25:52.263537 27938 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:52.263772 27938 heartbeater.cc:511] Master 127.21.81.254:43881 requested a full tablet report, sending...
I20260809 19:25:52.264364 27736 ts_manager.cc:194] Registered new tserver with Master: a252ea660bb34ed78174c4bb298d83b0 (127.21.81.193:35995)
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
I20260809 19:25:52.265080 27736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:46705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
W20260809 19:25:52.974723 21831 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.81.194: real 0.719s user 0.000s sys 0.004s
I20260809 19:25:52.975082 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43881
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:53.057675 27944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:53.057936 27944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:53.057978 27944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:53.059207 27944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:53.059283 27944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:53.059329 27944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:25:53.060875 27944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:53.061172 27944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:53.061425 27944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:53.066464 27950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:53.066476 27951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:53.066478 27953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:53.066699 27944 server_base.cc:1034] running on GCE node
I20260809 19:25:53.066938 27944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:53.067193 27944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:53.068363 27944 hybrid_clock.cc:648] HybridClock initialized: now 1786303553068351 us; error 32 us; skew 500 ppm
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[27689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303553, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:53.071738 27944 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:25:53.072368 27944 webserver.cc:460] Webserver started at http://127.21.81.194:40859/ using document root <none> and password file <none>
I20260809 19:25:53.072686 27944 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:53.072768 27944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:53.072909 27944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:53.074455 27944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "ac1b6e29f3d349da830e2ea4dc3a44c6"
format_stamp: "Formatted at 2026-08-09 19:25:53 on dist-test-slave-njxd"
server_key: "ab14f8b42c9cffe26978172f0d444488"
server_key_iv: "d214c6b7f22eb6bbd5299d6d87fe48aa"
server_key_version: "encryptionkey@0"
I20260809 19:25:53.074908 27944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "ac1b6e29f3d349da830e2ea4dc3a44c6"
format_stamp: "Formatted at 2026-08-09 19:25:53 on dist-test-slave-njxd"
server_key: "ab14f8b42c9cffe26978172f0d444488"
server_key_iv: "d214c6b7f22eb6bbd5299d6d87fe48aa"
server_key_version: "encryptionkey@0"
I20260809 19:25:53.076779 27944 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 19:25:53.077668 27960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:53.077934 27944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:53.078038 27944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "ac1b6e29f3d349da830e2ea4dc3a44c6"
format_stamp: "Formatted at 2026-08-09 19:25:53 on dist-test-slave-njxd"
server_key: "ab14f8b42c9cffe26978172f0d444488"
server_key_iv: "d214c6b7f22eb6bbd5299d6d87fe48aa"
server_key_version: "encryptionkey@0"
I20260809 19:25:53.078118 27944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:53.086199 27944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:53.087939 27944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:53.088106 27944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:53.088338 27944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:53.088660 27944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:53.088719 27944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:53.088761 27944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:53.088788 27944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:53.094918 27944 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:45807
I20260809 19:25:53.094975 28083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:45807 every 8 connection(s)
I20260809 19:25:53.095322 27944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:25:53.100595 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 27944
I20260809 19:25:53.100723 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:25:53.100930 21831 external_mini_cluster.cc:1428] Setting key 813ed29e06b6d5c843523d05276e6ea2
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[27689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303553, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:53.107087 28086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:53.096057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:55747 (local address 127.21.81.254:43881)
0809 19:25:53.097307 (+ 1250us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:53.097308 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:53.097316 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:53.097340 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:53.097340 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:53.097352 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:53.097404 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:53.098142 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:53.098668 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:53.099703 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:53.099889 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:53.099937 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:53.102518 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:53.102519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:53.102520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:53.102526 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:53.105295 (+ 2769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:53.105896 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:53.105897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:53.105897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:53.105958 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:53.106269 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:53.106270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:53.106270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:53.106385 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:53.106437 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:53.106781 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 19:25:53.107014 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1228,"thread_start_us":77,"threads_started":1}
I20260809 19:25:53.107250 28087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:53.096035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:43881 (local address 127.21.81.194:55747)
0809 19:25:53.096203 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:53.096249 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:53.097150 (+ 901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:53.097496 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:53.097499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:53.097859 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:53.098060 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:53.098064 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:53.098772 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:53.098774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:53.099627 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:53.099629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:53.099959 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:53.099960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:53.100056 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:53.100562 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:53.100571 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:53.102418 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:53.105389 (+ 2971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:53.105397 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:53.105399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:53.105804 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:53.106031 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:53.106032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:53.106033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:53.106190 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:53.106452 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:53.106458 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:53.106635 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 19:25:53.106756 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 19:25:53.106959 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260809 19:25:53.107913 28084 heartbeater.cc:347] Connected to a master server at 127.21.81.254:43881
I20260809 19:25:53.108021 28084 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:53.108238 28084 heartbeater.cc:511] Master 127.21.81.254:43881 requested a full tablet report, sending...
I20260809 19:25:53.108690 27736 ts_manager.cc:194] Registered new tserver with Master: ac1b6e29f3d349da830e2ea4dc3a44c6 (127.21.81.194:45807)
I20260809 19:25:53.109076 27736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:55747
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:25:53.129083 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43881
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:53.214879 28090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:53.215095 28090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:53.215122 28090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:53.216387 28090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:53.216435 28090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:53.216463 28090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:25:53.218060 28090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:53.218319 28090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:53.218559 28090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:53.223665 28096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:53.223713 28099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:53.223677 28095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:53.223713 28090 server_base.cc:1034] running on GCE node
I20260809 19:25:53.224102 28090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:53.224387 28090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:53.225551 28090 hybrid_clock.cc:648] HybridClock initialized: now 1786303553225539 us; error 48 us; skew 500 ppm
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[27689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303553, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:53.228148 28090 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:25:53.228639 28090 webserver.cc:460] Webserver started at http://127.21.81.195:35961/ using document root <none> and password file <none>
I20260809 19:25:53.228870 28090 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:53.228940 28090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:53.229095 28090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:53.230329 28090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "159f75cebca94ce0afd6013af74c1e93"
format_stamp: "Formatted at 2026-08-09 19:25:53 on dist-test-slave-njxd"
server_key: "8542d1ecc4201666cb73adb2cf694db8"
server_key_iv: "d36f7217940ebde764db13662227cdb5"
server_key_version: "encryptionkey@0"
I20260809 19:25:53.230715 28090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "159f75cebca94ce0afd6013af74c1e93"
format_stamp: "Formatted at 2026-08-09 19:25:53 on dist-test-slave-njxd"
server_key: "8542d1ecc4201666cb73adb2cf694db8"
server_key_iv: "d36f7217940ebde764db13662227cdb5"
server_key_version: "encryptionkey@0"
I20260809 19:25:53.232080 28090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:25:53.232918 28106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:53.233103 28090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:53.233212 28090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "159f75cebca94ce0afd6013af74c1e93"
format_stamp: "Formatted at 2026-08-09 19:25:53 on dist-test-slave-njxd"
server_key: "8542d1ecc4201666cb73adb2cf694db8"
server_key_iv: "d36f7217940ebde764db13662227cdb5"
server_key_version: "encryptionkey@0"
I20260809 19:25:53.233287 28090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:53.240218 28090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:53.242022 28090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:53.242188 28090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:53.242421 28090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:53.242738 28090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:53.242791 28090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:53.242832 28090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:53.242861 28090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:53.248947 28090 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:35839
I20260809 19:25:53.248997 28228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:35839 every 8 connection(s)
I20260809 19:25:53.249313 28090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:25:53.254801 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 28090
I20260809 19:25:53.254925 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:25:53.255147 21831 external_mini_cluster.cc:1428] Setting key af68fbc6ee0a3c4ce1598798e5436792
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[27689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303553, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:25:53.259254 28086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:53.250098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:53385 (local address 127.21.81.254:43881)
0809 19:25:53.250168 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:53.250170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:53.250421 (+ 251us) server_negotiation.cc:407] Connection header received
0809 19:25:53.251320 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:53.251321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:53.251332 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:53.251395 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:53.252234 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:53.252761 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:53.253926 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:53.254143 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:53.254205 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:53.256749 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:53.256750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:53.256750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:53.256757 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:53.258025 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:53.258452 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:53.258453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:53.258453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:53.258506 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:53.258730 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:53.258731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:53.258731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:53.258835 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:53.258886 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:53.259116 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 19:25:53.259190 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 19:25:53.259454 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:53.250079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:43881 (local address 127.21.81.195:53385)
0809 19:25:53.250257 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:53.250302 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:53.251229 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:53.251446 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:53.251449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:53.251848 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:53.252127 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:53.252134 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:53.252875 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:53.252877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:53.253843 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:53.253846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:53.254225 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:53.254225 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:53.254326 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:53.254851 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:53.254862 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:53.256666 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:53.258133 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:53.258140 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:53.258142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:53.258379 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:53.258567 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:53.258568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:53.258568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:53.258674 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:53.258892 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:53.258897 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:53.259009 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 19:25:53.259097 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 19:25:53.259221 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260809 19:25:53.259960 28229 heartbeater.cc:347] Connected to a master server at 127.21.81.254:43881
I20260809 19:25:53.260051 28229 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:53.260197 28229 heartbeater.cc:511] Master 127.21.81.254:43881 requested a full tablet report, sending...
I20260809 19:25:53.260568 27736 ts_manager.cc:194] Registered new tserver with Master: 159f75cebca94ce0afd6013af74c1e93 (127.21.81.195:35839)
I20260809 19:25:53.260910 27736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:53385
I20260809 19:25:53.265980 27938 heartbeater.cc:503] Master 127.21.81.254:43881 was elected leader, sending a full tablet report...
I20260809 19:25:53.266893 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/test-admin.keytab.
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[27689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:53.282196 21831 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 19:25:53.283933 28236 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:53.283126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45090 (local address 127.21.81.193:35995)
0809 19:25:53.283304 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:53.283308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:53.283318 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:25:53.283344 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:53.283346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:53.283359 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:53.283420 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:53.283826 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45090: BlockingRecv error: recv got EOF from 127.0.0.1:45090 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
W20260809 19:25:53.284055 21831 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/krb5cc' of type FILE
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[27689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:53.285167 21831 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[27689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:53.286149 21831 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[27689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.193@KRBTEST.COM
I20260809 19:25:53.293515 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:53.286656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45096 (local address 127.21.81.193:35995)
0809 19:25:53.286698 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:53.286699 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:53.286707 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:53.286790 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:53.286790 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:53.286799 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:53.286850 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:53.287220 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:53.287914 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:53.288623 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:53.288947 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:53.289011 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:53.291165 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:53.291173 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:53.291173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:53.291180 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:53.292539 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:53.292867 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:53.292870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:53.292870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:53.292923 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:53.293097 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:53.293098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:53.293098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:53.293221 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:53.293331 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:53.293388 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 19:25:53.293450 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 19:25:53.295361 28236 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:53.294767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45104 (local address 127.21.81.193:35995)
0809 19:25:53.294801 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:53.294803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:53.294899 (+ 96us) server_negotiation.cc:407] Connection header received
0809 19:25:53.294956 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:53.294957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:53.294967 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:53.295038 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:53.295319 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45104: BlockingRecv error: recv got EOF from 127.0.0.1:45104 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 19:25:53.295646 21831 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/krb5cc)
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[27689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:53.296607 21831 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[27689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:53.297700 21831 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[27689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.193@KRBTEST.COM
I20260809 19:25:53.304549 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:53.298200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45116 (local address 127.21.81.193:35995)
0809 19:25:53.298254 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:53.298257 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:53.298327 (+ 70us) server_negotiation.cc:407] Connection header received
0809 19:25:53.298403 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:53.298404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:53.298413 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:53.298479 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:53.298854 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:53.299335 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:53.300065 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:53.300273 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:53.300348 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:53.302295 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:53.302296 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:53.302297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:53.302304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:53.303451 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:53.303794 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:53.303795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:53.303796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:53.303854 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:53.304052 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:53.304053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:53.304053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:53.304159 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:53.304257 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:53.304376 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 19:25:53.304451 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 19:25:53.306015 28236 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:53.305497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45126 (local address 127.21.81.193:35995)
0809 19:25:53.305532 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:53.305534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:53.305544 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:25:53.305662 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:53.305663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:53.305670 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:53.305715 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:53.305974 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45126: BlockingRecv error: recv got EOF from 127.0.0.1:45126 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 19:25:53.306119 21831 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303525464116-21831-0/krb5kdc/krb5cc' of type FILE
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[27689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:53.307018 21831 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[27689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:53.307904 21831 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[27689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.193@KRBTEST.COM
I20260809 19:25:53.314774 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:53.308426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45138 (local address 127.21.81.193:35995)
0809 19:25:53.308486 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:53.308488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:53.308501 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:25:53.308583 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:53.308583 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:53.308592 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:53.308667 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:53.309021 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:53.309459 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:53.310156 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:53.310348 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:53.310422 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:53.312365 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:53.312366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:53.312367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:53.312374 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:53.313622 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:53.313979 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:53.313980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:53.313980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:53.314050 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:53.314266 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:53.314267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:53.314267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:53.314369 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:53.314460 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:53.314594 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 19:25:53.314680 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 19:25:53.315385 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 27801
I20260809 19:25:53.323261 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 27944
I20260809 19:25:53.329252 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 28090
I20260809 19:25:53.335640 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 27702
2026-08-09T19:25:53Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3453 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[28249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[28249](info): set up 2 sockets
Aug 09 19:25:53 dist-test-slave-njxd krb5kdc[28249](info): commencing operation
krb5kdc: starting...
W20260809 19:25:54.401471 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:25:54 dist-test-slave-njxd krb5kdc[28249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:54Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.254.keytab.
Entry for principal oryx/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.254.keytab.
I20260809 19:25:54.526103 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:42869
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.254.keytab
--principal=oryx/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:42869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:54.608971 28263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:54.609174 28263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:54.609196 28263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:54.610353 28263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:54.610397 28263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:54.610411 28263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:54.610424 28263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:54.610435 28263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:54.611943 28263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:42869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.254.keytab
--principal=oryx/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:42869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:54.612140 28263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:54.612313 28263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:54.617260 28271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:54.617293 28268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:54.617324 28263 server_base.cc:1034] running on GCE node
W20260809 19:25:54.617293 28269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:54.617754 28263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:54.618063 28263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:54.619210 28263 hybrid_clock.cc:648] HybridClock initialized: now 1786303554619198 us; error 28 us; skew 500 ppm
Aug 09 19:25:54 dist-test-slave-njxd krb5kdc[28249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303554, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:54.621904 28263 init.cc:377] Logged in from keytab as oryx/127.21.81.254@KRBTEST.COM (short username oryx)
I20260809 19:25:54.622306 28263 webserver.cc:460] Webserver started at http://127.21.81.254:36263/ using document root <none> and password file <none>
I20260809 19:25:54.622501 28263 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:54.622542 28263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:54.622644 28263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:54.623656 28263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "62c195d9d60c46f08e956f8b6e3e88d6"
format_stamp: "Formatted at 2026-08-09 19:25:54 on dist-test-slave-njxd"
server_key: "cf1f7dc8a096d03f9d5e03cabd000978"
server_key_iv: "c37b38ea947cc023ae872b222378db05"
server_key_version: "encryptionkey@0"
I20260809 19:25:54.623965 28263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "62c195d9d60c46f08e956f8b6e3e88d6"
format_stamp: "Formatted at 2026-08-09 19:25:54 on dist-test-slave-njxd"
server_key: "cf1f7dc8a096d03f9d5e03cabd000978"
server_key_iv: "c37b38ea947cc023ae872b222378db05"
server_key_version: "encryptionkey@0"
I20260809 19:25:54.625367 28263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:25:54.626271 28279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:54.626426 28263 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:54.626518 28263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "62c195d9d60c46f08e956f8b6e3e88d6"
format_stamp: "Formatted at 2026-08-09 19:25:54 on dist-test-slave-njxd"
server_key: "cf1f7dc8a096d03f9d5e03cabd000978"
server_key_iv: "c37b38ea947cc023ae872b222378db05"
server_key_version: "encryptionkey@0"
I20260809 19:25:54.626571 28263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:54.655679 28263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:54.658140 28263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:54.658265 28263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:54.662395 28263 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:42869
I20260809 19:25:54.662472 28334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:42869 every 8 connection(s)
I20260809 19:25:54.662843 28263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:54.663926 28335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:54.666314 28335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6: Bootstrap starting.
I20260809 19:25:54.666917 28335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:54.667219 28335 log.cc:824] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:54.667943 28335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6: No bootstrap required, opened a new log
I20260809 19:25:54.669557 28335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } }
I20260809 19:25:54.669750 28335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:54.669798 28335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62c195d9d60c46f08e956f8b6e3e88d6, State: Initialized, Role: FOLLOWER
I20260809 19:25:54.669970 28335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } }
I20260809 19:25:54.670074 28335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:54.670135 28335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:54.670186 28335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:54.671005 28335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } }
I20260809 19:25:54.671134 28335 leader_election.cc:302] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62c195d9d60c46f08e956f8b6e3e88d6; no voters:
I20260809 19:25:54.671320 28335 leader_election.cc:288] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:54.671360 28338 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:54.671553 28338 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [term 1 LEADER]: Becoming Leader. State: Replica: 62c195d9d60c46f08e956f8b6e3e88d6, State: Running, Role: LEADER
I20260809 19:25:54.671576 28335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:54.671653 28338 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } }
I20260809 19:25:54.671964 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 28263
I20260809 19:25:54.671994 28340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62c195d9d60c46f08e956f8b6e3e88d6. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } }
I20260809 19:25:54.672094 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:25:54.672102 28340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:54.672314 28345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:54.672338 21831 external_mini_cluster.cc:1428] Setting key e53557e28abcfa15b77429e0972a2352
I20260809 19:25:54.672870 28345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:54.673395 28339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } }
I20260809 19:25:54.673507 28339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:54.674623 28345 catalog_manager.cc:1347] Generated new cluster ID: f65849561df347f09a62f5020a8d2aa1
I20260809 19:25:54.674695 28345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:25:54 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
I20260809 19:25:54.684319 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:54.673014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59798 (local address 127.21.81.254:42869)
0809 19:25:54.673191 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:54.673202 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:54.673236 (+ 34us) server_negotiation.cc:407] Connection header received
0809 19:25:54.673663 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:54.673673 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:54.673745 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:54.673892 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:54.674691 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:54.676022 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:54.677107 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:54.677740 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:54.677850 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:54.680689 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:54.680694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:54.680695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:54.680710 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:54.682684 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:54.683057 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:54.683060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:54.683060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:54.683178 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:54.683379 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:54.683380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:54.683380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:54.683684 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:54.683868 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:54.683946 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 19:25:54.684096 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260809 19:25:54.698596 28345 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:54.699115 28345 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:54.701035 28345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6: Generated new TSK 0
I20260809 19:25:54.701213 28345 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.193.keytab.
Entry for principal oryx/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.193.keytab.
I20260809 19:25:54.728305 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42869
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.193.keytab
--principal=oryx/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:54.810916 28362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:54.811124 28362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:54.811153 28362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:54.812282 28362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:54.812323 28362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:54.812350 28362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:25:54.813984 28362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.193.keytab
--principal=oryx/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:54.814247 28362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:54.814484 28362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:54.819459 28367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:54.819471 28370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:54.819471 28368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:54.819665 28362 server_base.cc:1034] running on GCE node
I20260809 19:25:54.819916 28362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:54.820189 28362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:54.821358 28362 hybrid_clock.cc:648] HybridClock initialized: now 1786303554821350 us; error 39 us; skew 500 ppm
Aug 09 19:25:54 dist-test-slave-njxd krb5kdc[28249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303554, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:54.826746 28362 init.cc:377] Logged in from keytab as oryx/127.21.81.193@KRBTEST.COM (short username oryx)
I20260809 19:25:54.827227 28362 webserver.cc:460] Webserver started at http://127.21.81.193:37955/ using document root <none> and password file <none>
I20260809 19:25:54.827455 28362 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:54.827522 28362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:54.827678 28362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:54.828686 28362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "10c48d2cd2e04bc9a3c56487fea6b135"
format_stamp: "Formatted at 2026-08-09 19:25:54 on dist-test-slave-njxd"
server_key: "24ce5ba7004e5bfb7891738a7c1d323a"
server_key_iv: "51ab5773db0a6edce611d712f0171797"
server_key_version: "encryptionkey@0"
I20260809 19:25:54.829053 28362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "10c48d2cd2e04bc9a3c56487fea6b135"
format_stamp: "Formatted at 2026-08-09 19:25:54 on dist-test-slave-njxd"
server_key: "24ce5ba7004e5bfb7891738a7c1d323a"
server_key_iv: "51ab5773db0a6edce611d712f0171797"
server_key_version: "encryptionkey@0"
I20260809 19:25:54.830492 28362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:25:54.831216 28378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:54.831390 28362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:54.831509 28362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "10c48d2cd2e04bc9a3c56487fea6b135"
format_stamp: "Formatted at 2026-08-09 19:25:54 on dist-test-slave-njxd"
server_key: "24ce5ba7004e5bfb7891738a7c1d323a"
server_key_iv: "51ab5773db0a6edce611d712f0171797"
server_key_version: "encryptionkey@0"
I20260809 19:25:54.831579 28362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:54.852126 28362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:54.854348 28362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:54.854509 28362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:54.854769 28362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:54.855077 28362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:54.855129 28362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:54.855165 28362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:54.855211 28362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:54.861141 28362 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:44385
I20260809 19:25:54.861195 28498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:44385 every 8 connection(s)
I20260809 19:25:54.861486 28362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:25:54.863860 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 28362
I20260809 19:25:54.863943 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:25:54.864161 21831 external_mini_cluster.cc:1428] Setting key 0ee4718d2a6471d152bb59a056371810
Aug 09 19:25:54 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303554, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
I20260809 19:25:54.871726 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:54.862212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:55093 (local address 127.21.81.254:42869)
0809 19:25:54.862257 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:54.862259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:54.862495 (+ 236us) server_negotiation.cc:407] Connection header received
0809 19:25:54.862985 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:54.862986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:54.862998 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:54.863054 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:54.864002 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:54.864653 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:54.865745 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:54.865952 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:54.866030 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:54.868489 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:54.868491 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:54.868491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:54.868500 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:54.870188 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:54.870609 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:54.870611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:54.870611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:54.870679 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:54.870928 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:54.870929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:54.870929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:54.871127 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:54.871226 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:54.871490 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 19:25:54.871582 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:25:54.871830 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:54.862184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.193:55093)
0809 19:25:54.862367 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:54.862409 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:54.862903 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:54.863131 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:54.863134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:54.863683 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:54.863876 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:54.863881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:54.864764 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:54.864765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:54.865559 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:54.865561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:54.866039 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:54.866041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:54.866174 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:54.866719 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:54.866730 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:54.868397 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:54.870285 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:54.870292 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:54.870294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:54.870515 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:54.870756 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:54.870757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:54.870757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:54.870854 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:54.871228 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:54.871235 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:54.871351 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 19:25:54.871432 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 19:25:54.871608 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":127,"threads_started":1}
I20260809 19:25:54.872452 28499 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42869
I20260809 19:25:54.872567 28499 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:54.872754 28499 heartbeater.cc:511] Master 127.21.81.254:42869 requested a full tablet report, sending...
I20260809 19:25:54.873441 28298 ts_manager.cc:194] Registered new tserver with Master: 10c48d2cd2e04bc9a3c56487fea6b135 (127.21.81.193:44385)
WARNING: no policy specified for oryx/127.21.81.194@KRBTEST.COM; defaulting to no policy
I20260809 19:25:54.874425 28298 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:55093
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.194.keytab.
Entry for principal oryx/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.194.keytab.
I20260809 19:25:54.891749 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42869
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.194.keytab
--principal=oryx/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:54.973512 28504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:54.973734 28504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:54.973758 28504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:54.975345 28504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:54.975404 28504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:54.975448 28504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:25:54.977793 28504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.194.keytab
--principal=oryx/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:54.978053 28504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:54.978305 28504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:25:54.984510 28504 server_base.cc:1034] running on GCE node
W20260809 19:25:54.984558 28510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:54.984658 28509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:54.984683 28512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:54.984972 28504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:54.985257 28504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:54.986414 28504 hybrid_clock.cc:648] HybridClock initialized: now 1786303554986399 us; error 34 us; skew 500 ppm
Aug 09 19:25:54 dist-test-slave-njxd krb5kdc[28249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303554, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:54.989544 28504 init.cc:377] Logged in from keytab as oryx/127.21.81.194@KRBTEST.COM (short username oryx)
I20260809 19:25:54.990410 28504 webserver.cc:460] Webserver started at http://127.21.81.194:35799/ using document root <none> and password file <none>
I20260809 19:25:54.990633 28504 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:54.990716 28504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:54.990849 28504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:54.992286 28504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "d51eb4d4e6874518a2d659a44cfe7d4d"
format_stamp: "Formatted at 2026-08-09 19:25:54 on dist-test-slave-njxd"
server_key: "0a1888748f1b9771a8771452466f478f"
server_key_iv: "8115c155eb0539d411d35d9371be8b56"
server_key_version: "encryptionkey@0"
I20260809 19:25:54.992727 28504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "d51eb4d4e6874518a2d659a44cfe7d4d"
format_stamp: "Formatted at 2026-08-09 19:25:54 on dist-test-slave-njxd"
server_key: "0a1888748f1b9771a8771452466f478f"
server_key_iv: "8115c155eb0539d411d35d9371be8b56"
server_key_version: "encryptionkey@0"
I20260809 19:25:54.994417 28504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260809 19:25:54.995265 28519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:54.995436 28504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:54.995540 28504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "d51eb4d4e6874518a2d659a44cfe7d4d"
format_stamp: "Formatted at 2026-08-09 19:25:54 on dist-test-slave-njxd"
server_key: "0a1888748f1b9771a8771452466f478f"
server_key_iv: "8115c155eb0539d411d35d9371be8b56"
server_key_version: "encryptionkey@0"
I20260809 19:25:54.995626 28504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:55.024906 28504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:55.027323 28504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:55.027460 28504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:55.027734 28504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:55.028052 28504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:55.028087 28504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:55.028110 28504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:55.028151 28504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:55.034325 28504 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:41395
I20260809 19:25:55.034412 28636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:41395 every 8 connection(s)
I20260809 19:25:55.034745 28504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:25:55.037192 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 28504
I20260809 19:25:55.037299 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:25:55.037511 21831 external_mini_cluster.cc:1428] Setting key 2032a25ea531bd5b825d3e786c456da5
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303554, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
I20260809 19:25:55.045110 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.035441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.194:37863)
0809 19:25:55.035607 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.035644 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.036169 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.036411 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.036415 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.037028 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.037321 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.037329 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.038201 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.038202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.039005 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.039007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.039500 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.039501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.039631 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.040146 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.040157 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.041851 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.043773 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.043779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.043780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.043989 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.044246 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.044247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.044247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.044342 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.044645 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.044650 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.044757 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.044836 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.044951 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260809 19:25:55.045105 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.035428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:37863 (local address 127.21.81.254:42869)
0809 19:25:55.035494 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.035496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.035739 (+ 243us) server_negotiation.cc:407] Connection header received
0809 19:25:55.036282 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.036284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.036298 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.036348 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.037426 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.038100 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.039093 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.039410 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.039486 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.041943 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.041944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.041945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.041953 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.043679 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.044073 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.044074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.044075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.044142 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.044412 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.044413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.044413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.044571 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.044654 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.044889 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.044989 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:25:55.045639 28637 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42869
I20260809 19:25:55.045753 28637 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:55.045953 28637 heartbeater.cc:511] Master 127.21.81.254:42869 requested a full tablet report, sending...
I20260809 19:25:55.046384 28298 ts_manager.cc:194] Registered new tserver with Master: d51eb4d4e6874518a2d659a44cfe7d4d (127.21.81.194:41395)
I20260809 19:25:55.046756 28298 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:37863
WARNING: no policy specified for oryx/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.195.keytab.
Entry for principal oryx/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.195.keytab.
I20260809 19:25:55.065654 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42869
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.195.keytab
--principal=oryx/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:55.152434 28643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:55.152650 28643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:55.152674 28643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:55.153855 28643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:55.153900 28643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:55.153929 28643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:25:55.155426 28643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.195.keytab
--principal=oryx/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:55.155619 28643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:55.155813 28643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:55.160977 28649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:55.160962 28651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:55.161145 28643 server_base.cc:1034] running on GCE node
W20260809 19:25:55.160950 28648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:55.161405 28643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:55.161782 28643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:55.162955 28643 hybrid_clock.cc:648] HybridClock initialized: now 1786303555162943 us; error 25 us; skew 500 ppm
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303555, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:55.165547 28643 init.cc:377] Logged in from keytab as oryx/127.21.81.195@KRBTEST.COM (short username oryx)
I20260809 19:25:55.166057 28643 webserver.cc:460] Webserver started at http://127.21.81.195:41459/ using document root <none> and password file <none>
I20260809 19:25:55.166296 28643 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:55.166365 28643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:55.166525 28643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:55.167596 28643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "00d4d3cc2b034776ac860df5fa8c1cf2"
format_stamp: "Formatted at 2026-08-09 19:25:55 on dist-test-slave-njxd"
server_key: "6312d805aceef98706f04b235d5f3d04"
server_key_iv: "7316566c9e65d0bad242e7bb93f6360b"
server_key_version: "encryptionkey@0"
I20260809 19:25:55.168001 28643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "00d4d3cc2b034776ac860df5fa8c1cf2"
format_stamp: "Formatted at 2026-08-09 19:25:55 on dist-test-slave-njxd"
server_key: "6312d805aceef98706f04b235d5f3d04"
server_key_iv: "7316566c9e65d0bad242e7bb93f6360b"
server_key_version: "encryptionkey@0"
I20260809 19:25:55.169322 28643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:25:55.170126 28658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:55.170333 28643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:25:55.170436 28643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "00d4d3cc2b034776ac860df5fa8c1cf2"
format_stamp: "Formatted at 2026-08-09 19:25:55 on dist-test-slave-njxd"
server_key: "6312d805aceef98706f04b235d5f3d04"
server_key_iv: "7316566c9e65d0bad242e7bb93f6360b"
server_key_version: "encryptionkey@0"
I20260809 19:25:55.170516 28643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:55.191759 28643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:55.193733 28643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:55.193900 28643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:55.194135 28643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:55.194463 28643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:55.194516 28643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:55.194557 28643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:55.194597 28643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:55.200670 28643 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:44841
I20260809 19:25:55.200702 28780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:44841 every 8 connection(s)
I20260809 19:25:55.201050 28643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303555, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
I20260809 19:25:55.210101 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.201919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:42349 (local address 127.21.81.254:42869)
0809 19:25:55.201988 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.201990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.202175 (+ 185us) server_negotiation.cc:407] Connection header received
0809 19:25:55.202655 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.202656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.202666 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.202721 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.203654 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.204151 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.205098 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.205279 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.205331 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.207402 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.207403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.207403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.207418 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.208903 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.209254 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.209255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.209255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.209308 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.209517 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.209518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.209518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.209690 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.209745 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.209964 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.210027 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:25:55.210276 28783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.201893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.195:42349)
0809 19:25:55.202048 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.202090 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.202574 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.202788 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.202791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.203371 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.203557 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.203562 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.204235 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.204235 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.205016 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.205018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.205353 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.205353 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.205449 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.205933 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.205940 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.207296 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.208977 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.208982 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.208984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.209187 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.209368 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.209368 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.209368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.209463 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.209739 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.209743 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.209851 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.209932 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.210049 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260809 19:25:55.210872 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 28643
I20260809 19:25:55.210911 28781 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42869
I20260809 19:25:55.210985 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:25:55.211001 28781 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:55.211174 21831 external_mini_cluster.cc:1428] Setting key 4938f22f86c4d3ad2cda61097775172e
I20260809 19:25:55.211220 28781 heartbeater.cc:511] Master 127.21.81.254:42869 requested a full tablet report, sending...
I20260809 19:25:55.211634 28298 ts_manager.cc:194] Registered new tserver with Master: 00d4d3cc2b034776ac860df5fa8c1cf2 (127.21.81.195:44841)
I20260809 19:25:55.212119 28298 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:42349
I20260809 19:25:55.212136 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.81.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.81.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.81.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.253.keytab.
Entry for principal oryx/127.21.81.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.253.keytab.
I20260809 19:25:55.238634 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.253:45301
--webserver_interface=127.21.81.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.253.keytab
--principal=oryx/127.21.81.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:42869,127.21.81.253:45301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:55.321753 28787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:55.321966 28787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:55.321988 28787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:55.323256 28787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:55.323297 28787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:55.323312 28787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:55.323325 28787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:55.323338 28787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:55.324780 28787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:42869,127.21.81.253:45301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.253.keytab
--principal=oryx/127.21.81.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.253:45301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.81.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
W20260809 19:25:55.324875 28787 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.21.81.254:42869,127.21.81.253:45301'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 19:25:55.324990 28787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:55.325208 28787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:55.329988 28792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:55.329998 28793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:55.329998 28796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:55.330448 28787 server_base.cc:1034] running on GCE node
I20260809 19:25:55.330690 28787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:55.330945 28787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:55.332111 28787 hybrid_clock.cc:648] HybridClock initialized: now 1786303555332104 us; error 37 us; skew 500 ppm
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303555, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:55.334861 28787 init.cc:377] Logged in from keytab as oryx/127.21.81.253@KRBTEST.COM (short username oryx)
I20260809 19:25:55.335354 28787 webserver.cc:460] Webserver started at http://127.21.81.253:34349/ using document root <none> and password file <none>
I20260809 19:25:55.335609 28787 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:55.335681 28787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:55.335831 28787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:55.336908 28787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/data/instance:
uuid: "b333cac93a8642d884e8e6b1d6ca05c8"
format_stamp: "Formatted at 2026-08-09 19:25:55 on dist-test-slave-njxd"
server_key: "0cb53c2900f7451ac205c1a843dd8716"
server_key_iv: "9ed8e30779f303737349765f430c86f7"
server_key_version: "encryptionkey@0"
I20260809 19:25:55.337291 28787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/wal/instance:
uuid: "b333cac93a8642d884e8e6b1d6ca05c8"
format_stamp: "Formatted at 2026-08-09 19:25:55 on dist-test-slave-njxd"
server_key: "0cb53c2900f7451ac205c1a843dd8716"
server_key_iv: "9ed8e30779f303737349765f430c86f7"
server_key_version: "encryptionkey@0"
I20260809 19:25:55.338716 28787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:25:55.339449 28805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:55.339622 28787 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:55.339730 28787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/wal
uuid: "b333cac93a8642d884e8e6b1d6ca05c8"
format_stamp: "Formatted at 2026-08-09 19:25:55 on dist-test-slave-njxd"
server_key: "0cb53c2900f7451ac205c1a843dd8716"
server_key_iv: "9ed8e30779f303737349765f430c86f7"
server_key_version: "encryptionkey@0"
I20260809 19:25:55.339807 28787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:55.353072 28787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:55.355423 28787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:55.355602 28787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:55.359838 28787 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.253:45301
I20260809 19:25:55.359871 28858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.253:45301 every 8 connection(s)
I20260809 19:25:55.360227 28787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/data/info.pb
I20260809 19:25:55.361208 28859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:55.362835 28859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:25:55.363685 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 28787
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303555, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.253@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.81.253@KRBTEST.COM
I20260809 19:25:55.372570 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.363547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59812 (local address 127.21.81.254:42869)
0809 19:25:55.363642 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.363645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.363881 (+ 236us) server_negotiation.cc:407] Connection header received
0809 19:25:55.364490 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.364491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.364505 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.364560 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.365123 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.365676 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.366673 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.366871 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.366947 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.369301 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.369302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.369303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.369311 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.370918 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.371381 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.371381 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.371382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.371435 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.371709 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.371709 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.371710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.371923 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.372018 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.372324 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.372443 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:25:55.372681 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.363485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:59812)
0809 19:25:55.363723 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.363784 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.364381 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.364587 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.364591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.364863 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.365016 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.365020 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.365774 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.365775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.366577 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.366579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.366954 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.366955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.367075 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.367592 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.367610 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.369208 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.370996 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.371003 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.371006 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.371311 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.371500 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.371501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.371502 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.371639 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.371998 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.372002 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.372115 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.372249 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.372415 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":24,"thread_start_us":88,"threads_started":1}
I20260809 19:25:55.373378 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.364227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45936 (local address 127.21.81.253:45301)
0809 19:25:55.364377 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.364381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.364421 (+ 40us) server_negotiation.cc:407] Connection header received
0809 19:25:55.364546 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.364548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.364573 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.364658 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.365283 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.366178 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.366916 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.367244 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.367323 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.369868 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.369876 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.369876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.369883 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.371794 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.372166 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.372169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.372169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.372228 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.372527 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.372528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.372528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.372681 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.372830 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.373125 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.373219 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260809 19:25:55.373374 28859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303555, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.253@KRBTEST.COM for oryx/127.21.81.253@KRBTEST.COM
I20260809 19:25:55.380118 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.373662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:45301 (local address 127.0.0.1:45942)
0809 19:25:55.373849 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.373860 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.373911 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.374069 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.374069 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.374210 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.374255 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.374256 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.374907 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.374908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.375404 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.375406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.375732 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.375733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.375787 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.376179 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.376184 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.377299 (+ 1115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.379335 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.379336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.379336 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.379490 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.379672 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.379672 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.379672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.379721 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.379900 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.379900 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.379934 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.379983 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.380056 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172}
I20260809 19:25:55.380286 28864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.373702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45942 (local address 127.21.81.253:45301)
0809 19:25:55.373864 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.373866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.373896 (+ 30us) server_negotiation.cc:407] Connection header received
0809 19:25:55.373965 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.373966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.373976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.374033 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.374319 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.374837 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.375473 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.375671 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.375723 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.377373 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.377374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.377374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.377380 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.379267 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.379550 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.379550 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.379551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.379615 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.379766 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.379766 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.379767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.379852 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.379895 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.379997 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.380103 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":142,"thread_start_us":114,"threads_started":1}
I20260809 19:25:55.381502 28859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: Bootstrap starting.
I20260809 19:25:55.382184 28859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:55.382414 28859 log.cc:824] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:55.383127 28859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: No bootstrap required, opened a new log
I20260809 19:25:55.384467 28859 raft_consensus.cc:348] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } }
I20260809 19:25:55.384589 28859 raft_consensus.cc:374] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:55.384615 28859 raft_consensus.cc:729] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b333cac93a8642d884e8e6b1d6ca05c8, State: Initialized, Role: FOLLOWER
I20260809 19:25:55.384702 28859 consensus_queue.cc:260] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } }
I20260809 19:25:55.384931 28866 sys_catalog.cc:455] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } } }
I20260809 19:25:55.384994 28859 sys_catalog.cc:564] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:55.385017 28866 sys_catalog.cc:458] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:25:55.386729 28878 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:25:55.386795 28878 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:25:55.388378 28787 master_runner.cc:428] Detected that this master b333cac93a8642d884e8e6b1d6ca05c8 is joining an existing cluster
I20260809 19:25:55.388427 28787 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.81.253:45301
I20260809 19:25:55.396133 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.389637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59814 (local address 127.21.81.254:42869)
0809 19:25:55.389760 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.389763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.389953 (+ 190us) server_negotiation.cc:407] Connection header received
0809 19:25:55.390015 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.390016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.390027 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.390072 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.390419 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.390968 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.391735 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.391943 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.392017 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.393122 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.393123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.393124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.393131 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.394790 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.395294 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.395295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.395296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.395353 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.395577 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.395577 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.395578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.395693 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.395781 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.395964 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.396038 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 19:25:55.396147 28885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.389531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:59814)
0809 19:25:55.389895 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.389901 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.389956 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.390085 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.390086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.390291 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.390341 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.390343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.391053 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.391054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.391641 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.391643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.392029 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.392030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.392114 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.392582 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.392593 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.393010 (+ 417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.394891 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.394892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.394892 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.395201 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.395433 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.395434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.395434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.395506 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.395809 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.395810 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.395856 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.395936 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.396038 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":3,"thread_start_us":133,"threads_started":1}
I20260809 19:25:55.396509 28864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.389677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45946 (local address 127.21.81.253:45301)
0809 19:25:55.389712 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.389714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.389843 (+ 129us) server_negotiation.cc:407] Connection header received
0809 19:25:55.389874 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.389874 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.389885 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.389947 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.390408 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.391147 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.392163 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.392458 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.392497 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.393491 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.393492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.393492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.393498 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.395359 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.395749 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.395750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.395751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.395804 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.396020 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.396021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.396021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.396134 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.396176 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.396343 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.396423 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:25:55.396770 28884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.389530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:45301 (local address 127.0.0.1:45946)
0809 19:25:55.389765 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.389777 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.389838 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.389986 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.389987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.390241 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.390323 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.390325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.391239 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.391240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.392073 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.392076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.392562 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.392563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.392631 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.393034 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.393038 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.393399 (+ 361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.395443 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.395443 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.395444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.395677 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.395875 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.395876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.395876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.395944 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.396216 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.396217 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.396250 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.396587 (+ 337us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.396681 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"spinlock_wait_cycles":9088,"thread_start_us":144,"threads_started":1}
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303554, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.254@KRBTEST.COM for oryx/127.21.81.253@KRBTEST.COM
I20260809 19:25:55.406658 28864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.398903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45962 (local address 127.21.81.253:45301)
0809 19:25:55.398943 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.398945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.399127 (+ 182us) server_negotiation.cc:407] Connection header received
0809 19:25:55.399211 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.399212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.399220 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.399270 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.399786 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.400238 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.401360 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.401530 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.401645 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.403899 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.403900 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.403900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.403908 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.405349 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.405838 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.405839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.405840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.405900 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.406102 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.406103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.406103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.406213 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.406294 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.406479 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.406558 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:25:55.406694 28887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.398818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:45301 (local address 127.0.0.1:45962)
0809 19:25:55.399011 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.399058 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.399138 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.399276 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.399280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.399626 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.399687 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.399691 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.400316 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.400318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.401285 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.401288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.401649 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.401650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.401746 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.402267 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.402272 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.403801 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.405453 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.405462 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.405465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.405747 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.405968 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.405969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.405970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.406031 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.406287 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.406294 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.406349 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.406452 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.406572 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":104,"threads_started":1}
I20260809 19:25:55.407316 28298 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.81.253:45301: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8"
member_type: NON_VOTER
last_known_addr {
host: "127.21.81.253"
port: 45301
}
attrs {
promote: true
}
}
dest_uuid: "62c195d9d60c46f08e956f8b6e3e88d6"
cas_config_opid_index: -1
I20260809 19:25:55.407665 28298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } }
I20260809 19:25:55.408165 28890 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b333cac93a8642d884e8e6b1d6ca05c8 (127.21.81.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } } }
I20260809 19:25:55.408509 28832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:55.409509 28832 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 FOLLOWER]: Refusing update from remote peer 62c195d9d60c46f08e956f8b6e3e88d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 19:25:55.409893 28888 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:55.410552 28890 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.81.253:45301
I20260809 19:25:55.410879 28787 master.cc:537] Master@127.21.81.253:45301 shutting down...
I20260809 19:25:55.412106 28888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } } }
I20260809 19:25:55.412192 28888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:55.412230 28832 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b333cac93a8642d884e8e6b1d6ca05c8 (127.21.81.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } } }
I20260809 19:25:55.414022 28898 sys_catalog.cc:455] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } } }
I20260809 19:25:55.414351 28898 sys_catalog.cc:458] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: This master's current role is: LEARNER
I20260809 19:25:55.414453 28866 sys_catalog.cc:455] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62c195d9d60c46f08e956f8b6e3e88d6. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } } }
I20260809 19:25:55.414577 28866 sys_catalog.cc:458] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: This master's current role is: LEARNER
I20260809 19:25:55.414697 28893 sys_catalog.cc:455] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } } }
I20260809 19:25:55.414778 28893 sys_catalog.cc:458] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: This master's current role is: LEARNER
I20260809 19:25:55.415354 28787 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 LEARNER]: Raft consensus shutting down.
I20260809 19:25:55.415463 28787 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 LEARNER]: Raft consensus is shut down!
I20260809 19:25:55.415503 28787 tablet_replica.cc:330] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: stopping tablet replica
I20260809 19:25:55.428818 28787 master.cc:552] Master@127.21.81.253:45301 shutdown complete.
I20260809 19:25:55.428905 28787 master_runner.cc:305] Clearing existing system tablet
I20260809 19:25:55.429050 28787 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:55.429100 28787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:55.429863 28787 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:25:55.430744 28900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:55.430917 28787 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:55.430984 28787 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:55.431010 28787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/wal
uuid: "b333cac93a8642d884e8e6b1d6ca05c8"
format_stamp: "Formatted at 2026-08-09 19:25:55 on dist-test-slave-njxd"
server_key: "0cb53c2900f7451ac205c1a843dd8716"
server_key_iv: "9ed8e30779f303737349765f430c86f7"
server_key_version: "encryptionkey@0"
I20260809 19:25:55.431891 28787 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:55.433506 28787 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 19:25:55.433667 28787 log.cc:1196] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 19:25:55.433926 28787 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: Deleting consensus metadata
I20260809 19:25:55.434056 28787 master_runner.cc:315] Copying system tablet from 127.21.81.254:42869
I20260809 19:25:55.434726 28787 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: tablet copy: Beginning tablet copy session from remote peer at address 127.21.81.254:42869
I20260809 19:25:55.441186 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.435112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59816 (local address 127.21.81.254:42869)
0809 19:25:55.435172 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.435174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.435276 (+ 102us) server_negotiation.cc:407] Connection header received
0809 19:25:55.435333 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.435334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.435344 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.435389 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.435776 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.436273 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.437298 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.437491 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.437542 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.438651 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.438653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.438653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.438660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.440081 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.440475 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.440475 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.440476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.440529 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.440729 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.440730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.440730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.440835 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.440887 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.441050 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.441116 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:25:55.441259 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.435014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:59816)
0809 19:25:55.435160 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.435172 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.435250 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.435415 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.435416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.435629 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.435704 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.435706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.436349 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.436350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.437208 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.437211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.437562 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.437563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.437688 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.438137 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.438143 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.438560 (+ 417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.440165 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.440166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.440167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.440397 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.440592 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.440593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.440594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.440668 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.440900 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.440901 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.440948 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.441024 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.441136 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":90,"threads_started":1}
I20260809 19:25:55.441957 28319 tablet_copy_service.cc:140] P 62c195d9d60c46f08e956f8b6e3e88d6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b333cac93a8642d884e8e6b1d6ca05c8 ({username='oryx', principal='oryx/127.21.81.253@KRBTEST.COM'} at 127.0.0.1:59816)
I20260809 19:25:55.442072 28319 tablet_copy_service.cc:161] P 62c195d9d60c46f08e956f8b6e3e88d6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b333cac93a8642d884e8e6b1d6ca05c8 at {username='oryx', principal='oryx/127.21.81.253@KRBTEST.COM'} at 127.0.0.1:59816: session id = b333cac93a8642d884e8e6b1d6ca05c8-00000000000000000000000000000000
I20260809 19:25:55.442922 28319 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 19:25:55.443615 28787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 19:25:55.444931 28787 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: tablet copy: Starting download of 0 data blocks...
I20260809 19:25:55.445086 28787 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: tablet copy: Starting download of 1 WAL segments...
I20260809 19:25:55.446012 28787 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 19:25:55.446933 28319 tablet_copy_service.cc:342] P 62c195d9d60c46f08e956f8b6e3e88d6: Request end of tablet copy session b333cac93a8642d884e8e6b1d6ca05c8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.81.253@KRBTEST.COM'} at 127.0.0.1:59816
I20260809 19:25:55.446990 28319 tablet_copy_service.cc:433] P 62c195d9d60c46f08e956f8b6e3e88d6: ending tablet copy session b333cac93a8642d884e8e6b1d6ca05c8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b333cac93a8642d884e8e6b1d6ca05c8
I20260809 19:25:55.449692 28787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:55.450807 28907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:55.450865 28910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:55.451084 28908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:55.451192 28787 server_base.cc:1034] running on GCE node
I20260809 19:25:55.451290 28787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:55.451449 28787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:55.452571 28787 hybrid_clock.cc:648] HybridClock initialized: now 1786303555452560 us; error 29 us; skew 500 ppm
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303555, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:55.454010 28787 init.cc:377] Logged in from keytab as oryx/127.21.81.253@KRBTEST.COM (short username oryx)
I20260809 19:25:55.454303 28787 webserver.cc:460] Webserver started at http://127.21.81.253:34997/ using document root <none> and password file <none>
I20260809 19:25:55.454434 28787 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:55.454465 28787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:55.457674 28787 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260809 19:25:55.458401 28917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:55.458578 28787 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:55.458634 28787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/wal
uuid: "b333cac93a8642d884e8e6b1d6ca05c8"
format_stamp: "Formatted at 2026-08-09 19:25:55 on dist-test-slave-njxd"
server_key: "0cb53c2900f7451ac205c1a843dd8716"
server_key_iv: "9ed8e30779f303737349765f430c86f7"
server_key_version: "encryptionkey@0"
I20260809 19:25:55.458717 28787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:55.476248 28787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:55.478153 28787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:55.481410 28787 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.253:45301
I20260809 19:25:55.481725 28973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.253:45301 every 8 connection(s)
I20260809 19:25:55.481818 28787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-1/data/info.pb
I20260809 19:25:55.482573 28974 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:25:55.483031 28974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: Bootstrap starting.
I20260809 19:25:55.484835 28974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:25:55.485080 28974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: Bootstrap complete.
I20260809 19:25:55.485293 28974 raft_consensus.cc:348] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } }
I20260809 19:25:55.485365 28974 raft_consensus.cc:729] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b333cac93a8642d884e8e6b1d6ca05c8, State: Initialized, Role: LEARNER
I20260809 19:25:55.485394 28974 consensus_queue.cc:260] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } }
I20260809 19:25:55.485661 28974 sys_catalog.cc:564] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:55.485631 28978 sys_catalog.cc:455] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } } }
I20260809 19:25:55.485785 28978 sys_catalog.cc:458] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: This master's current role is: LEARNER
I20260809 19:25:55.487169 28991 catalog_manager.cc:1259] Loaded cluster ID: f65849561df347f09a62f5020a8d2aa1
I20260809 19:25:55.487246 28991 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: loading cluster ID for follower catalog manager: success
I20260809 19:25:55.488497 28991 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: acquiring CA information for follower catalog manager: success
I20260809 19:25:55.488792 28991 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:25:55.533032 28992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.525953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45992 (local address 127.21.81.253:45301)
0809 19:25:55.526120 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.526122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.526130 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:55.526154 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.526155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.526164 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.526244 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.526944 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.527632 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.528414 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.528642 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.528745 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.530200 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.530201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.530202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.530211 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.531814 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.532198 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.532199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.532200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.532271 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.532481 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.532482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.532483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.532607 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.532714 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.532822 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.532917 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for oryx/127.21.81.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.81.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.81.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.252.keytab.
Entry for principal oryx/127.21.81.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.252.keytab.
I20260809 19:25:55.559906 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.252:41405
--webserver_interface=127.21.81.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.252.keytab
--principal=oryx/127.21.81.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:42869,127.21.81.253:45301,127.21.81.252:41405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:55.646036 28995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:55.646332 28995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:55.646373 28995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:55.647639 28995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:55.647748 28995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:55.647781 28995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:55.647815 28995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:55.647838 28995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:55.649473 28995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:42869,127.21.81.253:45301,127.21.81.252:41405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.252.keytab
--principal=oryx/127.21.81.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.252:41405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.81.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:55.649819 28995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:55.650081 28995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:55.655316 29001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:55.655360 28995 server_base.cc:1034] running on GCE node
W20260809 19:25:55.655316 29000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:55.655321 29004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:55.655774 28995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:55.656113 28995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:55.657276 28995 hybrid_clock.cc:648] HybridClock initialized: now 1786303555657263 us; error 43 us; skew 500 ppm
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303555, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:55.660347 28995 init.cc:377] Logged in from keytab as oryx/127.21.81.252@KRBTEST.COM (short username oryx)
I20260809 19:25:55.660786 28995 webserver.cc:460] Webserver started at http://127.21.81.252:46553/ using document root <none> and password file <none>
I20260809 19:25:55.660977 28995 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:55.661018 28995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:55.661123 28995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:55.662505 28995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/data/instance:
uuid: "9720c6304069425a9b0f6fe82ac6c2cb"
format_stamp: "Formatted at 2026-08-09 19:25:55 on dist-test-slave-njxd"
server_key: "525e3fb14e824211160f2418beb28483"
server_key_iv: "ddf5622c4ec5fc52718bae54957cdf61"
server_key_version: "encryptionkey@0"
I20260809 19:25:55.662957 28995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/wal/instance:
uuid: "9720c6304069425a9b0f6fe82ac6c2cb"
format_stamp: "Formatted at 2026-08-09 19:25:55 on dist-test-slave-njxd"
server_key: "525e3fb14e824211160f2418beb28483"
server_key_iv: "ddf5622c4ec5fc52718bae54957cdf61"
server_key_version: "encryptionkey@0"
I20260809 19:25:55.664515 28995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:25:55.665343 29011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:55.665547 28995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:55.665690 28995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/wal
uuid: "9720c6304069425a9b0f6fe82ac6c2cb"
format_stamp: "Formatted at 2026-08-09 19:25:55 on dist-test-slave-njxd"
server_key: "525e3fb14e824211160f2418beb28483"
server_key_iv: "ddf5622c4ec5fc52718bae54957cdf61"
server_key_version: "encryptionkey@0"
I20260809 19:25:55.665782 28995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:55.676841 28995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:55.678948 28995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:55.679069 28995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:55.685322 28995 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.252:41405
I20260809 19:25:55.685322 29068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.252:41405 every 8 connection(s)
I20260809 19:25:55.685842 28995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/data/info.pb
I20260809 19:25:55.687237 29069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:55.689040 29069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:25:55.695617 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 28995
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303555, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.252@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
I20260809 19:25:55.699672 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.689874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59824 (local address 127.21.81.254:42869)
0809 19:25:55.689979 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.689982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.690172 (+ 190us) server_negotiation.cc:407] Connection header received
0809 19:25:55.690722 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.690723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.690737 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.690802 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.691448 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.691959 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.693492 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.693830 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.693908 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.696497 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.696498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.696498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.696506 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.698187 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.698648 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.698650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.698651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.698721 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.698963 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.698964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.698964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.699105 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.699172 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.699482 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.699593 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 19:25:55.699896 29071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.689734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:59824)
0809 19:25:55.690027 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.690078 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.690569 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.690874 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.690877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.691148 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.691346 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.691350 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.692058 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.692059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.693391 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.693394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.693932 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.693933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.694075 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.694622 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.694632 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.696382 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.698272 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.698279 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.698281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.698576 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.698793 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.698793 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.698793 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.698898 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.699170 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.699175 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.699312 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.699442 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.699618 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":200,"threads_started":1}
I20260809 19:25:55.700670 29069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.81.252@KRBTEST.COM
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303555, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.252@KRBTEST.COM for oryx/127.21.81.253@KRBTEST.COM
I20260809 19:25:55.706115 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.696237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40808 (local address 127.21.81.252:41405)
0809 19:25:55.696459 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.696464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.696485 (+ 21us) server_negotiation.cc:407] Connection header received
0809 19:25:55.696507 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.696509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.696523 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.696631 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.697431 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.698347 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.699321 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.699848 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.699938 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.702833 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.702840 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.702841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.702847 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.704637 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.705049 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.705051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.705052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.705103 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.705358 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.705359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.705360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.705547 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.705750 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.705834 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.705895 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
I20260809 19:25:55.708204 29071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.700971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:45301 (local address 127.0.0.1:46002)
0809 19:25:55.701022 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.701032 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.701069 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.701232 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.701234 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.701399 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.701472 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.701474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.702246 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.702247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.703013 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.703015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.703394 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.703395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.703550 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.703958 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.703962 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.705524 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.707392 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.707393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.707393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.707551 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.707737 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.707737 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.707737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.707788 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.707991 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.707991 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.708025 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.708074 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.708146 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 19:25:55.708235 28992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.701046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46002 (local address 127.21.81.253:45301)
0809 19:25:55.701090 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.701091 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.701117 (+ 26us) server_negotiation.cc:407] Connection header received
0809 19:25:55.701151 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.701152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.701164 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.701205 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.701550 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.702140 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.703104 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.703319 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.703398 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.705680 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.705682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.705682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.705691 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.707323 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.707620 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.707621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.707621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.707670 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.707841 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.707841 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.707841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.707938 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.707987 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.708092 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.708172 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:25:55.708712 29069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303555, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.252@KRBTEST.COM for oryx/127.21.81.252@KRBTEST.COM
I20260809 19:25:55.714907 29071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.708955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:41405 (local address 127.0.0.1:40822)
0809 19:25:55.709006 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.709017 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.709059 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.709180 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.709181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.709317 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.709344 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.709345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.710060 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.710061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.710535 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.710537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.710949 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.710950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.711000 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.711342 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.711346 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.712467 (+ 1121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.713894 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.713895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.713895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.714061 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.714272 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.714273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.714274 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.714327 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.714579 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.714580 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.714617 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.714681 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.714786 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":4}
I20260809 19:25:55.714908 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.709008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40822 (local address 127.21.81.252:41405)
0809 19:25:55.709041 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.709042 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.709050 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:55.709123 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.709124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.709134 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.709180 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.709422 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.709980 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.710620 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.710865 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.710941 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.712555 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.712557 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.712557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.712564 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.713803 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.714151 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.714152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.714153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.714207 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.714397 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.714398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.714399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.714502 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.714587 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.714699 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.714834 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":18}
I20260809 19:25:55.716425 29069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: Bootstrap starting.
I20260809 19:25:55.717291 29069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:55.717661 29069 log.cc:824] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:55.718434 29069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: No bootstrap required, opened a new log
I20260809 19:25:55.720347 29069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } }
I20260809 19:25:55.720481 29069 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:55.720503 29069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9720c6304069425a9b0f6fe82ac6c2cb, State: Initialized, Role: FOLLOWER
I20260809 19:25:55.720715 29069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } }
I20260809 19:25:55.721007 29074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } } }
I20260809 19:25:55.721071 29069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:55.721131 29074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:25:55.723263 29086 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:25:55.723326 29086 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:25:55.725800 28995 master_runner.cc:428] Detected that this master 9720c6304069425a9b0f6fe82ac6c2cb is joining an existing cluster
I20260809 19:25:55.725867 28995 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.81.252:41405
I20260809 19:25:55.734751 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.727213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:41405 (local address 127.0.0.1:40828)
0809 19:25:55.727645 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.727654 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.727714 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.727862 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.727863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.728304 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.728372 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.728374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.729195 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.729196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.730246 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.730249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.730755 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.730756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.730810 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.731400 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.731404 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.731771 (+ 367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.733546 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.733547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.733548 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.733805 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.734065 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.734066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.734066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.734134 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.734439 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.734440 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.734491 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.734548 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.734672 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":136,"threads_started":1}
I20260809 19:25:55.734848 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.727422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40828 (local address 127.21.81.252:41405)
0809 19:25:55.727464 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.727466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.727696 (+ 230us) server_negotiation.cc:407] Connection header received
0809 19:25:55.727775 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.727775 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.727783 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.727808 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.728455 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.729109 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.730341 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.730638 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.730675 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.731861 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.731862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.731862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.731869 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.733462 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.733896 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.733897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.733898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.733979 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.734210 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.734211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.734212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.734352 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.734440 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.734616 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.734740 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":23}
I20260809 19:25:55.750355 28992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.727317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46012 (local address 127.21.81.253:45301)
0809 19:25:55.727381 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.727383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.727532 (+ 149us) server_negotiation.cc:407] Connection header received
0809 19:25:55.727642 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.727643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.727655 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.727707 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.728381 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.729221 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.730526 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.730862 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.730910 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.747224 (+ 16314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.747226 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.747227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.747237 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.749016 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.749420 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.749422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.749422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.749492 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.749745 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.749747 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.749747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.749902 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.749988 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.750159 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.750249 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:25:55.750464 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.727029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59834 (local address 127.21.81.254:42869)
0809 19:25:55.727089 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.727091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.727222 (+ 131us) server_negotiation.cc:407] Connection header received
0809 19:25:55.727356 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.727357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.727370 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.727404 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.728178 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.729072 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.730657 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.730973 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.731018 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.747054 (+ 16036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.747056 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.747057 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.747073 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.748941 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.749475 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.749477 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.749477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.749538 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.749810 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.749812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.749812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.749966 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.750032 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.750268 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.750362 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:25:55.750418 29094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.727020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:45301 (local address 127.0.0.1:46012)
0809 19:25:55.727481 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.727491 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.727554 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.727760 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.727761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.728113 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.728276 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.728279 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.729379 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.729380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.730407 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.730410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.730992 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.730993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.731169 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.746489 (+ 15320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.746495 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.747000 (+ 505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.749103 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.749104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.749105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.749337 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.749565 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.749565 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.749566 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.749664 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.749977 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.749978 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.750030 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.750117 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.750248 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":182,"threads_started":1}
I20260809 19:25:55.750991 29092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.726992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:59834)
0809 19:25:55.727157 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.727168 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.727252 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.727480 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.727481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.727866 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.728046 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.728049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.729599 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.729600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.730566 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.730567 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.731114 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.731115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.731233 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.746380 (+ 15147us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.746389 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.746908 (+ 519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.749063 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.749065 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.749065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.749394 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.749644 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.749645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.749646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.749708 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.750061 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.750063 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.750111 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.750581 (+ 470us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.750704 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260809 19:25:55.757539 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 28362
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303554, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.254@KRBTEST.COM for oryx/127.21.81.252@KRBTEST.COM
I20260809 19:25:55.761389 28887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.753433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:41405 (local address 127.0.0.1:40838)
0809 19:25:55.753546 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.753557 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.753647 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.753803 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.753804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.753939 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.753985 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.753986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.754997 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.754998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.755529 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.755530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.756016 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.756017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.756092 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.756546 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.756554 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.758131 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.760144 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.760145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.760146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.760392 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.760685 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.760686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.760686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.760750 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.761035 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.761036 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.761084 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.761144 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.761275 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 19:25:55.761838 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.753494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40838 (local address 127.21.81.252:41405)
0809 19:25:55.753540 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.753542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.753630 (+ 88us) server_negotiation.cc:407] Connection header received
0809 19:25:55.753725 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.753726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.753737 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.753801 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.754069 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.754899 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.755621 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.755922 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.755996 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.758230 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.758231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.758231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.758238 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.760037 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.760480 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.760481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.760482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.760613 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.760816 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.760817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.760818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.760953 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.761009 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.761659 (+ 650us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.761746 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:25:55.762538 28298 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.81.252:41405: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb"
member_type: NON_VOTER
last_known_addr {
host: "127.21.81.252"
port: 41405
}
attrs {
promote: true
}
}
dest_uuid: "62c195d9d60c46f08e956f8b6e3e88d6"
cas_config_opid_index: 5
I20260809 19:25:55.762887 28298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } }
I20260809 19:25:55.763375 28888 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9720c6304069425a9b0f6fe82ac6c2cb (127.21.81.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } } }
I20260809 19:25:55.765048 28888 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.81.252:41405
I20260809 19:25:55.765154 28888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } } }
I20260809 19:25:55.765268 28888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:55.766129 28995 master.cc:537] Master@127.21.81.252:41405 shutting down...
I20260809 19:25:55.766238 29042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:55.767596 29042 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 1 FOLLOWER]: Refusing update from remote peer 62c195d9d60c46f08e956f8b6e3e88d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 19:25:55.767966 28890 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 19:25:55.768793 28281 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 19:25:55.772030 28995 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:55.772153 28995 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:55.772205 28995 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: stopping tablet replica
I20260809 19:25:55.774556 28887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.765539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:45301 (local address 127.0.0.1:46026)
0809 19:25:55.768807 (+ 3268us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.768821 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.768895 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.769204 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.769206 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.769208 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:55.769264 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.769267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.771551 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.771552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.773171 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.773176 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.774195 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.774196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.774284 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.774310 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.774437 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.774483 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3242}
I20260809 19:25:55.775017 28992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.765653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46026 (local address 127.21.81.253:45301)
0809 19:25:55.765736 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.765738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.768967 (+ 3229us) server_negotiation.cc:407] Connection header received
0809 19:25:55.769005 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.769007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.769041 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.769121 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:55.769389 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.771440 (+ 2051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.773318 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.773994 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.774060 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.774149 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.774889 (+ 740us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.774945 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 19:25:55.774637 28281 consensus_peers.cc:487] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 -> Peer 9720c6304069425a9b0f6fe82ac6c2cb (127.21.81.252:41405): Couldn't send request to peer 9720c6304069425a9b0f6fe82ac6c2cb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 19:25:55.775521 28946 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 LEARNER]: Refusing update from remote peer 62c195d9d60c46f08e956f8b6e3e88d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 19:25:55.775743 28890 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 19:25:55.776198 28946 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9720c6304069425a9b0f6fe82ac6c2cb (127.21.81.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } } }
I20260809 19:25:55.777391 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:44385
--local_ip_for_outbound_sockets=127.21.81.193
--tserver_master_addrs=127.21.81.254:42869
--webserver_port=37955
--webserver_interface=127.21.81.193
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.193.keytab
--principal=oryx/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.81.254:42869,127.21.81.253:45301
--tserver_master_addrs=127.21.81.254:42869,127.21.81.253:45301,127.21.81.252:41405 with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:25:55.777374 28978 sys_catalog.cc:455] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62c195d9d60c46f08e956f8b6e3e88d6. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } } }
I20260809 19:25:55.777508 28978 sys_catalog.cc:458] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: This master's current role is: LEARNER
I20260809 19:25:55.777549 28978 sys_catalog.cc:455] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: NON_VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: true } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } } }
I20260809 19:25:55.777670 28978 sys_catalog.cc:458] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: This master's current role is: LEARNER
I20260809 19:25:55.777891 28890 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6: attempting to promote NON_VOTER b333cac93a8642d884e8e6b1d6ca05c8 to VOTER
I20260809 19:25:55.778092 28890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } }
I20260809 19:25:55.778878 28946 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 LEARNER]: Refusing update from remote peer 62c195d9d60c46f08e956f8b6e3e88d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 19:25:55.779042 28888 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260809 19:25:55.780133 28281 connection.cc:535] client connection to 127.21.81.252:41405 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 19:25:55.780200 28281 consensus_peers.cc:487] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 -> Peer 9720c6304069425a9b0f6fe82ac6c2cb (127.21.81.252:41405): Couldn't send request to peer 9720c6304069425a9b0f6fe82ac6c2cb. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20260809 19:25:55.780172 29099 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b333cac93a8642d884e8e6b1d6ca05c8 (127.21.81.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } } }
I20260809 19:25:55.781536 28946 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b333cac93a8642d884e8e6b1d6ca05c8 (127.21.81.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } } }
I20260809 19:25:55.782593 29098 sys_catalog.cc:455] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } } }
I20260809 19:25:55.782717 29098 sys_catalog.cc:458] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:55.782965 28888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } } }
I20260809 19:25:55.783053 28888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:55.783185 29099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } } }
I20260809 19:25:55.783243 29099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:55.791884 28995 master.cc:552] Master@127.21.81.252:41405 shutdown complete.
I20260809 19:25:55.791950 28995 master_runner.cc:305] Clearing existing system tablet
I20260809 19:25:55.792085 28995 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:55.792135 28995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:55.792924 28995 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:55.793510 29103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:55.793720 28995 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:55.793772 28995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:25:55.793794 28995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/wal
uuid: "9720c6304069425a9b0f6fe82ac6c2cb"
format_stamp: "Formatted at 2026-08-09 19:25:55 on dist-test-slave-njxd"
server_key: "525e3fb14e824211160f2418beb28483"
server_key_iv: "ddf5622c4ec5fc52718bae54957cdf61"
server_key_version: "encryptionkey@0"
I20260809 19:25:55.794688 28995 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:55.796269 28995 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 19:25:55.796350 28995 log.cc:1196] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 19:25:55.796548 28995 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: Deleting consensus metadata
I20260809 19:25:55.796693 28995 master_runner.cc:315] Copying system tablet from 127.21.81.254:42869
I20260809 19:25:55.797335 28995 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: tablet copy: Beginning tablet copy session from remote peer at address 127.21.81.254:42869
I20260809 19:25:55.807034 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.797902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59840 (local address 127.21.81.254:42869)
0809 19:25:55.797957 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.797959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.799953 (+ 1994us) server_negotiation.cc:407] Connection header received
0809 19:25:55.800019 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.800020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.800035 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.800121 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.800538 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.801346 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.802437 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.802757 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.802829 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.803952 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.803954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.803954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.803963 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.805697 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.806170 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.806171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.806172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.806241 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.806463 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.806463 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.806464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.806608 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.806681 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.806856 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.806943 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:25:55.807050 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.797822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:59840)
0809 19:25:55.799846 (+ 2024us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.799862 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.799948 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.800141 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.800142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.800382 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.800449 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.800451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.801447 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.801448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.802334 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.802337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.802827 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.802828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.802888 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.803427 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.803433 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.803847 (+ 414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.805790 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.805792 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.805792 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.806084 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.806320 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.806320 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.806321 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.806392 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.806686 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.806687 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.806739 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.806815 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.806925 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1996,"thread_start_us":70,"threads_started":1}
I20260809 19:25:55.807752 28319 tablet_copy_service.cc:140] P 62c195d9d60c46f08e956f8b6e3e88d6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9720c6304069425a9b0f6fe82ac6c2cb ({username='oryx', principal='oryx/127.21.81.252@KRBTEST.COM'} at 127.0.0.1:59840)
I20260809 19:25:55.807825 28319 tablet_copy_service.cc:161] P 62c195d9d60c46f08e956f8b6e3e88d6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9720c6304069425a9b0f6fe82ac6c2cb at {username='oryx', principal='oryx/127.21.81.252@KRBTEST.COM'} at 127.0.0.1:59840: session id = 9720c6304069425a9b0f6fe82ac6c2cb-00000000000000000000000000000000
I20260809 19:25:55.808452 28319 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 19:25:55.809041 28995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 19:25:55.810577 28995 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: tablet copy: Starting download of 0 data blocks...
I20260809 19:25:55.810845 28995 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: tablet copy: Starting download of 1 WAL segments...
I20260809 19:25:55.811833 28995 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 19:25:55.812909 28319 tablet_copy_service.cc:342] P 62c195d9d60c46f08e956f8b6e3e88d6: Request end of tablet copy session 9720c6304069425a9b0f6fe82ac6c2cb-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.81.252@KRBTEST.COM'} at 127.0.0.1:59840
I20260809 19:25:55.812983 28319 tablet_copy_service.cc:433] P 62c195d9d60c46f08e956f8b6e3e88d6: ending tablet copy session 9720c6304069425a9b0f6fe82ac6c2cb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9720c6304069425a9b0f6fe82ac6c2cb
I20260809 19:25:55.816038 28995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:55.817261 29111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:55.817349 29112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:55.817747 29114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:55.821792 28995 server_base.cc:1034] running on GCE node
I20260809 19:25:55.821913 28995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:55.822074 28995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:55.823207 28995 hybrid_clock.cc:648] HybridClock initialized: now 1786303555823195 us; error 37 us; skew 500 ppm
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303555, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:55.824858 28995 init.cc:377] Logged in from keytab as oryx/127.21.81.252@KRBTEST.COM (short username oryx)
I20260809 19:25:55.825237 28995 webserver.cc:460] Webserver started at http://127.21.81.252:38379/ using document root <none> and password file <none>
I20260809 19:25:55.825385 28995 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:55.825433 28995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:55.826359 28995 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:25:55.826982 29121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:55.827167 28995 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:55.827222 28995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/wal
uuid: "9720c6304069425a9b0f6fe82ac6c2cb"
format_stamp: "Formatted at 2026-08-09 19:25:55 on dist-test-slave-njxd"
server_key: "525e3fb14e824211160f2418beb28483"
server_key_iv: "ddf5622c4ec5fc52718bae54957cdf61"
server_key_version: "encryptionkey@0"
I20260809 19:25:55.827286 28995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:55.839146 28995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:55.841135 28995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:55.844745 28995 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.252:41405
I20260809 19:25:55.844971 29176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.252:41405 every 8 connection(s)
I20260809 19:25:55.845122 28995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/master-2/data/info.pb
I20260809 19:25:55.846061 29177 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:25:55.846705 29177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: Bootstrap starting.
I20260809 19:25:55.849967 29177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:25:55.850409 29177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: Bootstrap complete.
I20260809 19:25:55.850699 29177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } }
I20260809 19:25:55.850805 29177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9720c6304069425a9b0f6fe82ac6c2cb, State: Initialized, Role: LEARNER
I20260809 19:25:55.850925 29177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } }
I20260809 19:25:55.851194 29177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:55.851207 29180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } } }
I20260809 19:25:55.851413 29180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [sys.catalog]: This master's current role is: LEARNER
I20260809 19:25:55.853503 29192 catalog_manager.cc:1259] Loaded cluster ID: f65849561df347f09a62f5020a8d2aa1
I20260809 19:25:55.853612 29192 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: loading cluster ID for follower catalog manager: success
I20260809 19:25:55.854776 29192 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: acquiring CA information for follower catalog manager: success
I20260809 19:25:55.855109 29192 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 19:25:55.882417 29100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:55.882642 29100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:55.882668 29100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:55.883939 29100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:55.883986 29100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:55.884014 29100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:25:55.885768 29100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.193.keytab
--principal=oryx/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:44385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=37955
--tserver_master_addrs=127.21.81.254:42869,127.21.81.253:45301,127.21.81.252:41405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:55.886022 29100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:55.886278 29100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:55.892546 29199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:55.892741 29201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:55.892603 29198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:55.893396 29100 server_base.cc:1034] running on GCE node
I20260809 19:25:55.893649 29100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:55.893945 29100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:55.895123 29100 hybrid_clock.cc:648] HybridClock initialized: now 1786303555895103 us; error 35 us; skew 500 ppm
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303555, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:55.898038 29100 init.cc:377] Logged in from keytab as oryx/127.21.81.193@KRBTEST.COM (short username oryx)
I20260809 19:25:55.898584 29100 webserver.cc:460] Webserver started at http://127.21.81.193:37955/ using document root <none> and password file <none>
I20260809 19:25:55.898850 29100 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:55.898909 29100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:55.900601 29100 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:25:55.901262 29208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:55.901475 29100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:55.901546 29100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "10c48d2cd2e04bc9a3c56487fea6b135"
format_stamp: "Formatted at 2026-08-09 19:25:54 on dist-test-slave-njxd"
server_key: "24ce5ba7004e5bfb7891738a7c1d323a"
server_key_iv: "51ab5773db0a6edce611d712f0171797"
server_key_version: "encryptionkey@0"
I20260809 19:25:55.901965 29100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:55.916651 29100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:55.918859 29100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:55.918984 29100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:55.919230 29100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:55.919504 29100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:55.919535 29100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:55.919567 29100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:55.919582 29100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:55.929294 29100 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:44385
I20260809 19:25:55.929342 29328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:44385 every 8 connection(s)
I20260809 19:25:55.929836 29100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:25:55.935043 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 29100
I20260809 19:25:55.935226 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 28504
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303555, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303555, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.253@KRBTEST.COM
Aug 09 19:25:55 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303555, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.252@KRBTEST.COM
I20260809 19:25:55.943691 28992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.930609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:34689 (local address 127.21.81.253:45301)
0809 19:25:55.930676 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.930679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.931017 (+ 338us) server_negotiation.cc:407] Connection header received
0809 19:25:55.932254 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.932256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.932269 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.932330 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.933218 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.934009 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.935472 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.935786 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.935831 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.940063 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.940064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.940065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.940074 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.942274 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.942719 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.942720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.942721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.942790 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.943016 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.943017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.943018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.943169 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.943217 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.943472 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.943577 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:25:55.943724 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.931130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:52649 (local address 127.21.81.254:42869)
0809 19:25:55.931239 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.931241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.931259 (+ 18us) server_negotiation.cc:407] Connection header received
0809 19:25:55.932298 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.932300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.932316 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.932348 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.933204 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.933805 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.935472 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.935786 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.935831 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.939964 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.939966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.939966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.939975 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.941941 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.942466 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.942468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.942469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.942533 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.942852 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.942854 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.942854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.943018 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.943067 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.943471 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.943568 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 19:25:55.943966 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:41395
--local_ip_for_outbound_sockets=127.21.81.194
--tserver_master_addrs=127.21.81.254:42869
--webserver_port=35799
--webserver_interface=127.21.81.194
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.194.keytab
--principal=oryx/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.81.254:42869,127.21.81.253:45301
--tserver_master_addrs=127.21.81.254:42869,127.21.81.253:45301,127.21.81.252:41405 with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:25:55.944332 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.930632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:45301 (local address 127.21.81.193:34689)
0809 19:25:55.930934 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.930957 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.932147 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.932407 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.932411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.932913 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.933095 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.933102 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.934102 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.934102 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.935355 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.935358 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.935932 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.935934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.936123 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.936940 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.936953 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.939955 (+ 3002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.942374 (+ 2419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.942375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.942376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.942614 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.942874 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.942875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.942876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.942942 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.943271 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.943271 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.943322 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.943846 (+ 524us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.944059 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":6,"thread_start_us":190,"threads_started":1}
I20260809 19:25:55.944681 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.930946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:41405 (local address 127.21.81.193:50909)
0809 19:25:55.931217 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.931228 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.932146 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.932432 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.932432 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.932917 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.933095 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.933102 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.934412 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.934413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.935356 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.935359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.938686 (+ 3327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.938688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.938828 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.939298 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.939303 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.941516 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.943586 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.943587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.943588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.943789 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.944016 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.944017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.944018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.944087 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.944372 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.944373 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.944415 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.944473 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.944583 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":6,"thread_start_us":59,"threads_started":1}
I20260809 19:25:55.945056 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.931233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:50909 (local address 127.21.81.252:41405)
0809 19:25:55.931417 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:55.931420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:55.931431 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:25:55.932333 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:55.932334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:55.932349 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:55.932427 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:55.933216 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.934300 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.938280 (+ 3980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.938605 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.938687 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.941646 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:55.941647 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:55.941648 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:55.941657 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:55.943498 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.943882 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.943883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.943883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.943943 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:55.944159 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:55.944159 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:55.944160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:55.944290 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:55.944866 (+ 576us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:55.944910 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 19:25:55.944991 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260809 19:25:55.945076 29333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:55.930632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.193:52649)
0809 19:25:55.930813 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:55.930857 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:55.932141 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:55.932427 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:55.932428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:55.932849 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:55.933094 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.933099 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.933922 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.933923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.935355 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:55.935358 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:55.935934 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:55.935935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:55.936123 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:55.937712 (+ 1589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:55.937717 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:55.939834 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:55.942037 (+ 2203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.942045 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.942048 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.942372 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.942618 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:55.942619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:55.942619 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:55.942766 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:55.943120 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:55.943127 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:55.943308 (+ 181us) client_negotiation.cc:769] Sending connection context
0809 19:25:55.944375 (+ 1067us) client_negotiation.cc:240] Negotiation successful
0809 19:25:55.944716 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":35,"thread_start_us":102,"threads_started":1}
I20260809 19:25:55.945676 29331 heartbeater.cc:347] Connected to a master server at 127.21.81.253:45301
I20260809 19:25:55.945784 29331 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:55.945807 29329 heartbeater.cc:347] Connected to a master server at 127.21.81.252:41405
I20260809 19:25:55.945864 29329 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:55.945996 29331 heartbeater.cc:511] Master 127.21.81.253:45301 requested a full tablet report, sending...
I20260809 19:25:55.945996 29329 heartbeater.cc:511] Master 127.21.81.252:41405 requested a full tablet report, sending...
I20260809 19:25:55.946249 29330 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42869
I20260809 19:25:55.946287 29330 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:55.946352 29330 heartbeater.cc:511] Master 127.21.81.254:42869 requested a full tablet report, sending...
I20260809 19:25:55.946616 28298 ts_manager.cc:194] Re-registered known tserver with Master: 10c48d2cd2e04bc9a3c56487fea6b135 (127.21.81.193:44385)
I20260809 19:25:55.946626 28936 ts_manager.cc:194] Registered new tserver with Master: 10c48d2cd2e04bc9a3c56487fea6b135 (127.21.81.193:44385)
I20260809 19:25:55.946646 29139 ts_manager.cc:194] Registered new tserver with Master: 10c48d2cd2e04bc9a3c56487fea6b135 (127.21.81.193:44385)
I20260809 19:25:55.947121 28298 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:52649
W20260809 19:25:56.033488 29337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:56.033751 29337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:56.033776 29337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:56.035044 29337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:56.035094 29337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:56.035121 29337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:25:56.036701 29337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.194.keytab
--principal=oryx/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:41395
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=35799
--tserver_master_addrs=127.21.81.254:42869,127.21.81.253:45301,127.21.81.252:41405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:56.036983 29337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:56.037253 29337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:56.042769 29342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:56.042814 29343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:56.042769 29345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:56.043114 29337 server_base.cc:1034] running on GCE node
I20260809 19:25:56.043334 29337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:56.043601 29337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:56.044750 29337 hybrid_clock.cc:648] HybridClock initialized: now 1786303556044733 us; error 41 us; skew 500 ppm
Aug 09 19:25:56 dist-test-slave-njxd krb5kdc[28249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303556, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:56.047417 29337 init.cc:377] Logged in from keytab as oryx/127.21.81.194@KRBTEST.COM (short username oryx)
I20260809 19:25:56.047914 29337 webserver.cc:460] Webserver started at http://127.21.81.194:35799/ using document root <none> and password file <none>
I20260809 19:25:56.048158 29337 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:56.048246 29337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:56.049738 29337 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:56.050465 29353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:56.050645 29337 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:56.050747 29337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "d51eb4d4e6874518a2d659a44cfe7d4d"
format_stamp: "Formatted at 2026-08-09 19:25:54 on dist-test-slave-njxd"
server_key: "0a1888748f1b9771a8771452466f478f"
server_key_iv: "8115c155eb0539d411d35d9371be8b56"
server_key_version: "encryptionkey@0"
I20260809 19:25:56.051122 29337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:56.073423 29337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:56.076181 29337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:56.076310 29337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:56.076502 29337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:56.076845 29337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:56.076884 29337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:56.076908 29337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:56.076956 29337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:56.083634 29337 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:41395
I20260809 19:25:56.083717 29471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:41395 every 8 connection(s)
I20260809 19:25:56.084076 29337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:25:56.090646 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 29337
I20260809 19:25:56.090835 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 28643
Aug 09 19:25:56 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303556, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.253@KRBTEST.COM
Aug 09 19:25:56 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303556, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
Aug 09 19:25:56 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303556, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.252@KRBTEST.COM
I20260809 19:25:56.098153 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.084877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:45767 (local address 127.21.81.254:42869)
0809 19:25:56.084944 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.084946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.085192 (+ 246us) server_negotiation.cc:407] Connection header received
0809 19:25:56.086330 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.086332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.086346 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.086398 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:56.087425 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.088041 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.089452 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.089742 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.089811 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.093442 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:56.093443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:56.093444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:56.093452 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:56.096267 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.096940 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.096942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.096942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.097004 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.097396 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.097397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.097397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.097522 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:56.097563 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.097965 (+ 402us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.098053 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 19:25:56.098156 28992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.084849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:48083 (local address 127.21.81.253:45301)
0809 19:25:56.084961 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.084963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.085218 (+ 255us) server_negotiation.cc:407] Connection header received
0809 19:25:56.086463 (+ 1245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.086464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.086476 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.086533 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:56.087446 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.088237 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.089288 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.089577 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.089671 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.094353 (+ 4682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:56.094355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:56.094355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:56.094364 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:56.096227 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.096941 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.096942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.096942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.097004 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.097342 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.097343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.097343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.097486 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:56.097532 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.097964 (+ 432us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.098053 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 19:25:56.098275 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.085266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:57551 (local address 127.21.81.252:41405)
0809 19:25:56.085313 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.085315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.085371 (+ 56us) server_negotiation.cc:407] Connection header received
0809 19:25:56.086596 (+ 1225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.086598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.086614 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.086667 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:56.087448 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.088216 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.089425 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.089800 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.089864 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.094310 (+ 4446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:56.094311 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:56.094312 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:56.094340 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:56.096565 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.097148 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.097149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.097150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.097203 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.097466 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.097468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.097468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.097679 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:56.097746 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.098008 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.098131 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:25:56.098918 29478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.085179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:41405 (local address 127.21.81.194:57551)
0809 19:25:56.085318 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:56.085328 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:56.086231 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:56.086720 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:56.086720 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:56.087041 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:56.087330 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.087337 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.088298 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.088299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.089339 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.089341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.089917 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.089918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.090526 (+ 608us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.091070 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:56.091076 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:56.094176 (+ 3100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:56.096673 (+ 2497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:56.096674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:56.096675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:56.097021 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:56.097280 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:56.097281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:56.097281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:56.097365 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:56.097804 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:56.097805 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:56.097862 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 19:25:56.098377 (+ 515us) client_negotiation.cc:240] Negotiation successful
0809 19:25:56.098704 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":282,"thread_start_us":81,"threads_started":1}
I20260809 19:25:56.098899 29477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.084859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:45301 (local address 127.21.81.194:48083)
0809 19:25:56.085047 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:56.085089 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:56.086226 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:56.086549 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:56.086549 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:56.087038 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:56.087330 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.087337 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.088324 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.088325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.089177 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.089179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.089677 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.089678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.089823 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.090368 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:56.090376 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:56.094220 (+ 3844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:56.096330 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:56.096339 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:56.096342 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:56.096843 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:56.097086 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:56.097086 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:56.097086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:56.097264 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:56.097598 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:56.097605 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:56.097782 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 19:25:56.098303 (+ 521us) client_negotiation.cc:240] Negotiation successful
0809 19:25:56.098603 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":17,"thread_start_us":110,"threads_started":1}
I20260809 19:25:56.100694 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.084858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.194:45767)
0809 19:25:56.085045 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:56.085088 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:56.086216 (+ 1128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:56.086484 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:56.086488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:56.087028 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:56.087329 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.087334 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.088146 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.088147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.089336 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.089339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.089826 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.089827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.089953 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.090540 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:56.090545 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:56.093334 (+ 2789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:56.096349 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:56.096350 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:56.096350 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:56.096842 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:56.097086 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:56.097086 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:56.097086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:56.097319 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:56.097655 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:56.097655 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:56.097782 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 19:25:56.098807 (+ 1025us) client_negotiation.cc:240] Negotiation successful
0809 19:25:56.098902 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":122,"thread_start_us":125,"threads_started":1}
I20260809 19:25:56.100693 29474 heartbeater.cc:347] Connected to a master server at 127.21.81.253:45301
I20260809 19:25:56.100936 29474 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:56.101118 29474 heartbeater.cc:511] Master 127.21.81.253:45301 requested a full tablet report, sending...
I20260809 19:25:56.101612 28936 ts_manager.cc:194] Registered new tserver with Master: d51eb4d4e6874518a2d659a44cfe7d4d (127.21.81.194:41395)
I20260809 19:25:56.102501 29472 heartbeater.cc:347] Connected to a master server at 127.21.81.252:41405
I20260809 19:25:56.102551 29472 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:56.102625 29472 heartbeater.cc:511] Master 127.21.81.252:41405 requested a full tablet report, sending...
I20260809 19:25:56.102878 29473 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42869
I20260809 19:25:56.102929 29473 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:56.102991 29473 heartbeater.cc:511] Master 127.21.81.254:42869 requested a full tablet report, sending...
I20260809 19:25:56.103001 29139 ts_manager.cc:194] Registered new tserver with Master: d51eb4d4e6874518a2d659a44cfe7d4d (127.21.81.194:41395)
I20260809 19:25:56.103312 28298 ts_manager.cc:194] Re-registered known tserver with Master: d51eb4d4e6874518a2d659a44cfe7d4d (127.21.81.194:41395)
I20260809 19:25:56.103871 28298 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:45767
I20260809 19:25:56.107368 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:44841
--local_ip_for_outbound_sockets=127.21.81.195
--tserver_master_addrs=127.21.81.254:42869
--webserver_port=41459
--webserver_interface=127.21.81.195
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.195.keytab
--principal=oryx/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.81.254:42869,127.21.81.253:45301
--tserver_master_addrs=127.21.81.254:42869,127.21.81.253:45301,127.21.81.252:41405 with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:25:56.190268 28887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.187442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:41405 (local address 127.0.0.1:40844)
0809 19:25:56.187495 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:56.187508 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:56.187585 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:56.187809 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:56.187810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:56.187812 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:56.187871 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.187873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.188568 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.188570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.189323 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.189325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.190022 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.190023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.190088 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.190093 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:25:56.190129 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 19:25:56.190187 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 19:25:56.190268 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.187516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40844 (local address 127.21.81.252:41405)
0809 19:25:56.187592 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.187595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.187608 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:25:56.187679 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.187680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.187719 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.187799 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:56.187957 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.188466 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.189425 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.189922 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.190020 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.190105 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.190154 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.190189 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 19:25:56.190627 29149 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 19:25:56.190853 29180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62c195d9d60c46f08e956f8b6e3e88d6. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } } }
I20260809 19:25:56.190995 29180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [sys.catalog]: This master's current role is: LEARNER
W20260809 19:25:56.194708 29479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:56.194943 29479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:56.194985 29479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:56.196228 29479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:56.196296 29479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:56.196343 29479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:25:56.198004 29479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.195.keytab
--principal=oryx/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:44841
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=41459
--tserver_master_addrs=127.21.81.254:42869,127.21.81.253:45301,127.21.81.252:41405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:56.198299 29479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:56.198544 29479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:56.203786 29487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:56.203810 29479 server_base.cc:1034] running on GCE node
W20260809 19:25:56.203809 29485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:56.203786 29484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:56.204339 29479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:56.204558 29479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:56.205706 29479 hybrid_clock.cc:648] HybridClock initialized: now 1786303556205692 us; error 29 us; skew 500 ppm
Aug 09 19:25:56 dist-test-slave-njxd krb5kdc[28249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303556, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:56.208457 29479 init.cc:377] Logged in from keytab as oryx/127.21.81.195@KRBTEST.COM (short username oryx)
I20260809 19:25:56.208940 29479 webserver.cc:460] Webserver started at http://127.21.81.195:41459/ using document root <none> and password file <none>
I20260809 19:25:56.209192 29479 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:56.209283 29479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:56.210845 29479 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:56.211512 29494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:56.211704 29479 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:56.211813 29479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "00d4d3cc2b034776ac860df5fa8c1cf2"
format_stamp: "Formatted at 2026-08-09 19:25:55 on dist-test-slave-njxd"
server_key: "6312d805aceef98706f04b235d5f3d04"
server_key_iv: "7316566c9e65d0bad242e7bb93f6360b"
server_key_version: "encryptionkey@0"
I20260809 19:25:56.212214 29479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:56.225224 29479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:56.228017 29479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:56.228205 29479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:56.228480 29479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:25:56.228816 29479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:25:56.228871 29479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:56.228917 29479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:25:56.228943 29479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:56.235404 29479 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:44841
I20260809 19:25:56.235488 29619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:44841 every 8 connection(s)
I20260809 19:25:56.235854 29479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:25:56.243084 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 29479
Aug 09 19:25:56 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303556, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
Aug 09 19:25:56 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303556, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.253@KRBTEST.COM
Aug 09 19:25:56 dist-test-slave-njxd krb5kdc[28249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303556, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.252@KRBTEST.COM
I20260809 19:25:56.250768 29627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.237178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:45301 (local address 127.21.81.195:41581)
0809 19:25:56.237299 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:56.237309 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:56.238198 (+ 889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:56.238478 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:56.238479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:56.238958 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:56.239216 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.239221 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.239951 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.239952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.241377 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.241380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.241917 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.241918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.242074 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.242877 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:56.242882 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:56.246615 (+ 3733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:56.248798 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:56.248808 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:56.248810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:56.249173 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:56.249437 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:56.249438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:56.249438 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:56.249643 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:56.249968 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:56.249975 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:56.250164 (+ 189us) client_negotiation.cc:769] Sending connection context
0809 19:25:56.250282 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 19:25:56.250488 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260809 19:25:56.250941 29625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.236966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:41405 (local address 127.21.81.195:59795)
0809 19:25:56.237149 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:56.237160 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:56.238173 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:56.238561 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:56.238562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:56.238987 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:56.239217 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.239223 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.241721 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.241722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.242342 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.242344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.242834 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.242836 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.242909 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.243328 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:56.243333 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:56.246237 (+ 2904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:56.249710 (+ 3473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:56.249711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:56.249711 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:56.249940 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:56.250164 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:56.250165 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:56.250165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:56.250228 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:56.250550 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:56.250551 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:56.250620 (+ 69us) client_negotiation.cc:769] Sending connection context
0809 19:25:56.250692 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 19:25:56.250804 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":6,"thread_start_us":83,"threads_started":1}
I20260809 19:25:56.251139 28992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.237276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:41581 (local address 127.21.81.253:45301)
0809 19:25:56.237325 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.237326 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.237379 (+ 53us) server_negotiation.cc:407] Connection header received
0809 19:25:56.238345 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.238347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.238359 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.238393 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:56.239322 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.239833 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.241477 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.241823 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.241896 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.246708 (+ 4812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:56.246710 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:56.246710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:56.246717 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:56.248707 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.249279 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.249280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.249280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.249358 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.249733 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.249734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.249735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.249875 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:56.249939 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.250956 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.251050 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:25:56.251363 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.237068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:59795 (local address 127.21.81.252:41405)
0809 19:25:56.237126 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.237129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.237217 (+ 88us) server_negotiation.cc:407] Connection header received
0809 19:25:56.238466 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.238467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.238483 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.238543 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:56.240904 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.241646 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.242409 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.242749 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.242826 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.246344 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:56.246345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:56.246346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:56.246354 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:56.249615 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.250025 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.250026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.250027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.250082 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.250295 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.250295 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.250296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.250441 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:56.250522 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.251163 (+ 641us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.251272 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:25:56.251513 29624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.236671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.195:33485)
0809 19:25:56.236867 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:56.236906 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:56.238173 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:56.238463 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:56.238466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:56.238986 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:56.239216 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.239223 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.240869 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.240869 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.241401 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.241402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.241999 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.242000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.242067 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.242729 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:56.242738 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:56.247660 (+ 4922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:56.250317 (+ 2657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:56.250317 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:56.250318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:56.250591 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:56.250801 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:56.250802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:56.250802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:56.250855 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:56.251161 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:56.251162 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:56.251219 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 19:25:56.251284 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 19:25:56.251384 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260809 19:25:56.251752 29620 heartbeater.cc:347] Connected to a master server at 127.21.81.252:41405
I20260809 19:25:56.251848 29620 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:56.252069 29620 heartbeater.cc:511] Master 127.21.81.252:41405 requested a full tablet report, sending...
I20260809 19:25:56.252455 29622 heartbeater.cc:347] Connected to a master server at 127.21.81.253:45301
I20260809 19:25:56.252506 29622 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:56.252575 29622 heartbeater.cc:511] Master 127.21.81.253:45301 requested a full tablet report, sending...
I20260809 19:25:56.252972 29139 ts_manager.cc:194] Registered new tserver with Master: 00d4d3cc2b034776ac860df5fa8c1cf2 (127.21.81.195:44841)
I20260809 19:25:56.252982 28936 ts_manager.cc:194] Registered new tserver with Master: 00d4d3cc2b034776ac860df5fa8c1cf2 (127.21.81.195:44841)
I20260809 19:25:56.253626 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.236677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:33485 (local address 127.21.81.254:42869)
0809 19:25:56.236760 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.236763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.237036 (+ 273us) server_negotiation.cc:407] Connection header received
0809 19:25:56.238313 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.238315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.238331 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.238403 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:56.239971 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.240784 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.241569 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.241879 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.241944 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.247754 (+ 5810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:56.247755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:56.247756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:56.247763 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:56.250227 (+ 2464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.250670 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.250672 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.250672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.250735 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.250917 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.250918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.250918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.251053 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:56.251120 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.253387 (+ 2267us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.253479 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:25:56.254074 29621 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42869
I20260809 19:25:56.254135 29621 heartbeater.cc:464] Registering TS with master...
I20260809 19:25:56.254206 29621 heartbeater.cc:511] Master 127.21.81.254:42869 requested a full tablet report, sending...
I20260809 19:25:56.254494 28298 ts_manager.cc:194] Re-registered known tserver with Master: 00d4d3cc2b034776ac860df5fa8c1cf2 (127.21.81.195:44841)
I20260809 19:25:56.255045 28298 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:33485
I20260809 19:25:56.255460 29637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.244615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40856 (local address 127.21.81.252:41405)
0809 19:25:56.244789 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.244791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.245027 (+ 236us) server_negotiation.cc:407] Connection header received
0809 19:25:56.245056 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.245057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.245070 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.245131 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:56.246342 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.247085 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.248179 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.248497 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.248561 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.250270 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:56.250272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:56.250272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:56.250278 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:56.254068 (+ 3790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.254467 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.254469 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.254469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.254530 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.254823 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.254824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.254824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.254941 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:56.255055 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.255261 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.255355 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":71,"threads_started":1}
I20260809 19:25:56.255939 29635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.244559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46036 (local address 127.21.81.253:45301)
0809 19:25:56.244850 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.244852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.245019 (+ 167us) server_negotiation.cc:407] Connection header received
0809 19:25:56.245046 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.245046 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.245059 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.245126 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:56.245915 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.246691 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.247823 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.248189 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.248260 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.252434 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:56.252435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:56.252436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:56.252443 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:56.254686 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.255053 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.255054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.255054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.255117 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.255365 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.255365 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.255366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.255501 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:56.255614 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.255802 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.255877 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":56,"threads_started":1}
I20260809 19:25:56.259872 29634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.244375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59856 (local address 127.21.81.254:42869)
0809 19:25:56.244857 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.244859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.244867 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:56.244893 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.244894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.244908 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.244977 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:56.245934 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.247199 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.253156 (+ 5957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.253502 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.253574 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.257431 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:56.257432 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:56.257433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:56.257438 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:56.258765 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.259092 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.259093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.259093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.259151 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.259346 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.259346 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.259347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.259467 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:56.259542 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.259730 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.259805 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":152,"threads_started":1}
I20260809 19:25:56.261358 28298 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:25:56.263254 28298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:25:56.264097 28888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: NON_VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: true } } }
I20260809 19:25:56.264215 28888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:56.265861 28888 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6: attempting to promote NON_VOTER 9720c6304069425a9b0f6fe82ac6c2cb to VOTER
I20260809 19:25:56.266006 28888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } }
I20260809 19:25:56.266429 28946 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 FOLLOWER]: Refusing update from remote peer 62c195d9d60c46f08e956f8b6e3e88d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260809 19:25:56.266645 29149 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 1 LEARNER]: Refusing update from remote peer 62c195d9d60c46f08e956f8b6e3e88d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 19:25:56.266701 28888 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 19:25:56.266848 29101 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 19:25:56.267612 28888 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9720c6304069425a9b0f6fe82ac6c2cb (127.21.81.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } } }
I20260809 19:25:56.267766 28946 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9720c6304069425a9b0f6fe82ac6c2cb (127.21.81.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } } }
I20260809 19:25:56.267784 29149 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9720c6304069425a9b0f6fe82ac6c2cb (127.21.81.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } } }
I20260809 19:25:56.268697 28888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } } }
I20260809 19:25:56.268800 28888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:56.268740 29180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } } }
I20260809 19:25:56.268847 29180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:56.268862 28888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } } }
I20260809 19:25:56.268918 28888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:56.268903 29098 sys_catalog.cc:455] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } } }
I20260809 19:25:56.268959 28888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } } }
I20260809 19:25:56.268990 29098 sys_catalog.cc:458] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:56.269003 28888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:56.274501 29646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.270060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:44841 (local address 127.0.0.1:51726)
0809 19:25:56.270267 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:56.270278 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:56.270328 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:56.270850 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:56.270851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:56.270853 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:56.270904 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.270906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.272135 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.272136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.273164 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.273167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.274261 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.274262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.274315 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.274319 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:56.274359 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 19:25:56.274394 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":71,"thread_start_us":149,"threads_started":1}
I20260809 19:25:56.274767 29651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.270366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51726 (local address 127.21.81.195:44841)
0809 19:25:56.270634 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.270638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.270647 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:25:56.270685 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.270687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.270704 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.270779 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:56.270987 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.272044 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.273255 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.274169 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.274252 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.274402 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.274436 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.274487 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":121,"threads_started":1}
I20260809 19:25:56.275162 28887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.269961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:41395 (local address 127.0.0.1:48500)
0809 19:25:56.269999 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:56.270011 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:56.270052 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:56.270849 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:56.270851 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:56.270853 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:56.270904 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.270906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.272335 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.272336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.273514 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.273515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.274488 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.274489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.274550 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.274554 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:56.275037 (+ 483us) client_negotiation.cc:240] Negotiation successful
0809 19:25:56.275094 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":44}
I20260809 19:25:56.275624 29546 tablet_service.cc:1467] Processing CreateTablet for tablet f82a2310c1ae47ee9e79faf8f289a7e1 (DEFAULT_TABLE table=test-table [id=131363fefff045b5be4082e403da6556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:56.275949 29647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.270084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:44385 (local address 127.0.0.1:40756)
0809 19:25:56.270422 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:56.270431 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:56.270471 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:56.270756 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:56.270757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:56.270759 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:56.270825 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.270827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.272314 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.272315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.273507 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.273510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.275725 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.275726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.275775 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.275779 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:56.275810 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 19:25:56.275846 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":11,"thread_start_us":112,"threads_started":1}
I20260809 19:25:56.276175 29546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f82a2310c1ae47ee9e79faf8f289a7e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:56.276253 29649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.270090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48500 (local address 127.21.81.194:41395)
0809 19:25:56.270593 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.270598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.270626 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:25:56.270732 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.270735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.270762 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.270895 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:56.270965 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.272197 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.273634 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.274399 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.274473 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.274548 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.276005 (+ 1457us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.276158 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":104,"threads_started":1}
I20260809 19:25:56.276728 29650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.270332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40756 (local address 127.21.81.193:44385)
0809 19:25:56.270531 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.270538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.270550 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:25:56.270596 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.270599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.270636 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.270739 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:56.270921 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.272235 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.274732 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.275623 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.275704 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.275897 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.276582 (+ 685us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.276641 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":120,"threads_started":1}
I20260809 19:25:56.277401 29259 tablet_service.cc:1467] Processing CreateTablet for tablet f82a2310c1ae47ee9e79faf8f289a7e1 (DEFAULT_TABLE table=test-table [id=131363fefff045b5be4082e403da6556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:56.277853 29259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f82a2310c1ae47ee9e79faf8f289a7e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:56.279942 29653 tablet_bootstrap.cc:492] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2: Bootstrap starting.
I20260809 19:25:56.280527 29405 tablet_service.cc:1467] Processing CreateTablet for tablet f82a2310c1ae47ee9e79faf8f289a7e1 (DEFAULT_TABLE table=test-table [id=131363fefff045b5be4082e403da6556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:56.280952 29653 tablet_bootstrap.cc:654] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:56.280982 29405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f82a2310c1ae47ee9e79faf8f289a7e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:56.281428 29653 log.cc:824] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:56.281412 29654 tablet_bootstrap.cc:492] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135: Bootstrap starting.
I20260809 19:25:56.282452 29653 tablet_bootstrap.cc:492] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2: No bootstrap required, opened a new log
I20260809 19:25:56.282536 29653 ts_tablet_manager.cc:1397] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:25:56.282541 29654 tablet_bootstrap.cc:654] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:56.283016 29654 log.cc:824] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:56.283808 29654 tablet_bootstrap.cc:492] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135: No bootstrap required, opened a new log
I20260809 19:25:56.283900 29654 ts_tablet_manager.cc:1397] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:25:56.285830 29657 tablet_bootstrap.cc:492] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d: Bootstrap starting.
I20260809 19:25:56.286054 29653 raft_consensus.cc:348] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } } peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } }
I20260809 19:25:56.286396 29653 raft_consensus.cc:374] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:56.286463 29653 raft_consensus.cc:729] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d4d3cc2b034776ac860df5fa8c1cf2, State: Initialized, Role: FOLLOWER
I20260809 19:25:56.286625 29653 consensus_queue.cc:260] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } } peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } }
I20260809 19:25:56.286844 29657 tablet_bootstrap.cc:654] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:56.286904 29653 ts_tablet_manager.cc:1428] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260809 19:25:56.287072 29621 heartbeater.cc:503] Master 127.21.81.254:42869 was elected leader, sending a full tablet report...
I20260809 19:25:56.287402 29657 log.cc:824] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:56.287745 29654 raft_consensus.cc:348] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } } peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } }
I20260809 19:25:56.287935 29654 raft_consensus.cc:374] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:56.287981 29654 raft_consensus.cc:729] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c48d2cd2e04bc9a3c56487fea6b135, State: Initialized, Role: FOLLOWER
I20260809 19:25:56.288115 29654 consensus_queue.cc:260] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } } peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } }
I20260809 19:25:56.288582 29654 ts_tablet_manager.cc:1428] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260809 19:25:56.288726 29330 heartbeater.cc:503] Master 127.21.81.254:42869 was elected leader, sending a full tablet report...
I20260809 19:25:56.288969 29657 tablet_bootstrap.cc:492] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d: No bootstrap required, opened a new log
I20260809 19:25:56.289038 29657 ts_tablet_manager.cc:1397] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:25:56.291599 29657 raft_consensus.cc:348] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } } peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } }
I20260809 19:25:56.291800 29657 raft_consensus.cc:374] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:56.291846 29657 raft_consensus.cc:729] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51eb4d4e6874518a2d659a44cfe7d4d, State: Initialized, Role: FOLLOWER
I20260809 19:25:56.291965 29657 consensus_queue.cc:260] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } } peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } }
I20260809 19:25:56.292215 29657 ts_tablet_manager.cc:1428] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260809 19:25:56.292341 29473 heartbeater.cc:503] Master 127.21.81.254:42869 was elected leader, sending a full tablet report...
W20260809 19:25:56.335083 29475 tablet.cc:2369] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:25:56.384434 29658 raft_consensus.cc:482] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:25:56.384588 29658 raft_consensus.cc:504] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } } peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } }
I20260809 19:25:56.384992 29658 leader_election.cc:288] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d51eb4d4e6874518a2d659a44cfe7d4d (127.21.81.194:41395), 10c48d2cd2e04bc9a3c56487fea6b135 (127.21.81.193:44385)
I20260809 19:25:56.388569 29649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.385424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:53887 (local address 127.21.81.194:41395)
0809 19:25:56.385484 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.385486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.385494 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:56.385540 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.385541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.385553 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.385640 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:56.385953 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.386614 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.387766 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.388123 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.388215 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.388224 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.388462 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.388508 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 19:25:56.388557 29624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.385212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:44385 (local address 127.21.81.195:56999)
0809 19:25:56.385303 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:56.385317 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:56.385368 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:56.385561 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:56.385562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:56.385564 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:56.385774 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.385777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.386690 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.386692 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.387635 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.387638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.388220 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.388221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.388340 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.388343 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:25:56.388389 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 19:25:56.388454 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":5}
I20260809 19:25:56.388547 29650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.385311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:56999 (local address 127.21.81.193:44385)
0809 19:25:56.385362 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.385364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.385378 (+ 14us) server_negotiation.cc:407] Connection header received
0809 19:25:56.385443 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.385444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.385455 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.385531 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:56.385867 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.386605 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.387743 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.388119 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.388214 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.388224 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.388425 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.388472 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:25:56.388615 29625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.385212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:41395 (local address 127.21.81.195:53887)
0809 19:25:56.385308 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:56.385317 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:56.385368 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:56.385722 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:56.385723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:56.385725 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:56.385874 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.385876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.386718 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.386719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.387641 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:56.387644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.388213 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.388214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:56.388353 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.388356 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:25:56.388436 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 19:25:56.388472 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":83}
I20260809 19:25:56.389019 29280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82a2310c1ae47ee9e79faf8f289a7e1" candidate_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" is_pre_election: true
I20260809 19:25:56.389043 29425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82a2310c1ae47ee9e79faf8f289a7e1" candidate_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" is_pre_election: true
I20260809 19:25:56.389215 29425 raft_consensus.cc:2454] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00d4d3cc2b034776ac860df5fa8c1cf2 in term 0.
I20260809 19:25:56.389215 29280 raft_consensus.cc:2454] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00d4d3cc2b034776ac860df5fa8c1cf2 in term 0.
I20260809 19:25:56.389460 29498 leader_election.cc:302] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d4d3cc2b034776ac860df5fa8c1cf2, d51eb4d4e6874518a2d659a44cfe7d4d; no voters:
I20260809 19:25:56.389683 29658 raft_consensus.cc:2781] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:25:56.389756 29658 raft_consensus.cc:482] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:25:56.389775 29658 raft_consensus.cc:3037] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:56.390611 29658 raft_consensus.cc:504] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } } peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } }
I20260809 19:25:56.390782 29658 leader_election.cc:288] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [CANDIDATE]: Term 1 election: Requested vote from peers d51eb4d4e6874518a2d659a44cfe7d4d (127.21.81.194:41395), 10c48d2cd2e04bc9a3c56487fea6b135 (127.21.81.193:44385)
I20260809 19:25:56.390944 29280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82a2310c1ae47ee9e79faf8f289a7e1" candidate_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10c48d2cd2e04bc9a3c56487fea6b135"
I20260809 19:25:56.391019 29280 raft_consensus.cc:3037] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:56.391942 29280 raft_consensus.cc:2454] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d4d3cc2b034776ac860df5fa8c1cf2 in term 1.
I20260809 19:25:56.392087 29425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82a2310c1ae47ee9e79faf8f289a7e1" candidate_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d"
I20260809 19:25:56.392127 29497 leader_election.cc:302] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d4d3cc2b034776ac860df5fa8c1cf2, 10c48d2cd2e04bc9a3c56487fea6b135; no voters:
I20260809 19:25:56.392203 29425 raft_consensus.cc:3037] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:56.392287 29658 raft_consensus.cc:2781] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:56.392508 29658 raft_consensus.cc:686] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 00d4d3cc2b034776ac860df5fa8c1cf2, State: Running, Role: LEADER
I20260809 19:25:56.392613 29658 consensus_queue.cc:237] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } } peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } }
I20260809 19:25:56.393146 29425 raft_consensus.cc:2454] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d4d3cc2b034776ac860df5fa8c1cf2 in term 1.
I20260809 19:25:56.393800 28298 catalog_manager.cc:5495] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00d4d3cc2b034776ac860df5fa8c1cf2 (127.21.81.195). New cstate: current_term: 1 leader_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } health_report { overall_health: UNKNOWN } } }
I20260809 19:25:56.420019 29651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.417553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51738 (local address 127.21.81.195:44841)
0809 19:25:56.417657 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.417659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.417680 (+ 21us) server_negotiation.cc:407] Connection header received
0809 19:25:56.417725 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.417726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.417745 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.417800 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:56.417931 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.418407 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.419178 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.419397 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.419476 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.419663 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:56.419773 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:56.419875 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.419895 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.419919 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 19:25:56.422122 29425 raft_consensus.cc:1261] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d [term 1 FOLLOWER]: Refusing update from remote peer 00d4d3cc2b034776ac860df5fa8c1cf2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:56.422123 29280 raft_consensus.cc:1261] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135 [term 1 FOLLOWER]: Refusing update from remote peer 00d4d3cc2b034776ac860df5fa8c1cf2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:56.422533 29663 consensus_queue.cc:1035] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:56.422685 29658 consensus_queue.cc:1035] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:56.427103 29667 mvcc.cc:205] Tried to move back new op lower bound from 7316699367101902848 to 7316699366984863744. Current Snapshot: MvccSnapshot[applied={T|T < 7316699367101902848 or (T in {7316699367101902848})}]
I20260809 19:25:56.426205 29666 mvcc.cc:205] Tried to move back new op lower bound from 7316699367101902848 to 7316699366984863744. Current Snapshot: MvccSnapshot[applied={T|T < 7316699367101902848}]
I20260809 19:25:56.427932 29668 mvcc.cc:205] Tried to move back new op lower bound from 7316699367101902848 to 7316699366984863744. Current Snapshot: MvccSnapshot[applied={T|T < 7316699367101902848 or (T in {7316699367101902848})}]
I20260809 19:25:56.430506 28298 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:56.430624 28298 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:25:56.432616 28298 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 62c195d9d60c46f08e956f8b6e3e88d6: Sending DeleteTablet for 3 replicas of tablet f82a2310c1ae47ee9e79faf8f289a7e1
I20260809 19:25:56.433081 29405 tablet_service.cc:1514] Processing DeleteTablet for tablet f82a2310c1ae47ee9e79faf8f289a7e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:56 UTC) from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:48500
I20260809 19:25:56.433050 29259 tablet_service.cc:1514] Processing DeleteTablet for tablet f82a2310c1ae47ee9e79faf8f289a7e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:56 UTC) from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:40756
I20260809 19:25:56.433444 29677 tablet_replica.cc:330] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135: stopping tablet replica
I20260809 19:25:56.433502 29546 tablet_service.cc:1514] Processing DeleteTablet for tablet f82a2310c1ae47ee9e79faf8f289a7e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:25:56 UTC) from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:51726
I20260809 19:25:56.433626 29677 raft_consensus.cc:2229] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:56.433930 29675 tablet_replica.cc:330] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d: stopping tablet replica
I20260809 19:25:56.433938 29678 tablet_replica.cc:330] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2: stopping tablet replica
I20260809 19:25:56.434053 29675 raft_consensus.cc:2229] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:25:56.434185 29675 raft_consensus.cc:2258] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:56.434193 29678 raft_consensus.cc:2229] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:25:56.434437 29678 raft_consensus.cc:2258] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:56.434685 29677 raft_consensus.cc:2258] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:25:56.435060 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 28263
I20260809 19:25:56.435114 29678 ts_tablet_manager.cc:1905] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:56.437860 29678 ts_tablet_manager.cc:1918] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:25:56.437934 29678 log.cc:1196] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/f82a2310c1ae47ee9e79faf8f289a7e1
I20260809 19:25:56.438226 29678 ts_tablet_manager.cc:1939] T f82a2310c1ae47ee9e79faf8f289a7e1 P 00d4d3cc2b034776ac860df5fa8c1cf2: Deleting consensus metadata
I20260809 19:25:56.442150 29677 ts_tablet_manager.cc:1905] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:56.443150 29675 ts_tablet_manager.cc:1905] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:25:56.444164 29677 ts_tablet_manager.cc:1918] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:25:56.444259 29677 log.cc:1196] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-0/wal/wals/f82a2310c1ae47ee9e79faf8f289a7e1
I20260809 19:25:56.444573 29677 ts_tablet_manager.cc:1939] T f82a2310c1ae47ee9e79faf8f289a7e1 P 10c48d2cd2e04bc9a3c56487fea6b135: Deleting consensus metadata
I20260809 19:25:56.446020 29675 ts_tablet_manager.cc:1918] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:25:56.446117 29675 log.cc:1196] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/f82a2310c1ae47ee9e79faf8f289a7e1
I20260809 19:25:56.446400 29675 ts_tablet_manager.cc:1939] T f82a2310c1ae47ee9e79faf8f289a7e1 P d51eb4d4e6874518a2d659a44cfe7d4d: Deleting consensus metadata
W20260809 19:25:56.447783 29212 connection.cc:535] server connection from 127.0.0.1:40756 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 19:25:56.447788 29500 connection.cc:535] server connection from 127.0.0.1:51726 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 19:25:56.448275 29354 connection.cc:535] server connection from 127.0.0.1:48500 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 19:25:56.450884 29637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.434513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40872 (local address 127.21.81.252:41405)
0809 19:25:56.434555 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.434557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.434931 (+ 374us) server_negotiation.cc:407] Connection header received
0809 19:25:56.435079 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.435080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.435095 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.435126 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:56.438626 (+ 3500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.439518 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.440635 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.440989 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.441079 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.446845 (+ 5766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:56.446847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:56.446848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:56.446859 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:56.448939 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.449405 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.449406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.449406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.449482 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.449758 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.449759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.449760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.449921 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:56.450031 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.450647 (+ 616us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.450761 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:25:56.459203 29635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.450312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46048 (local address 127.21.81.253:45301)
0809 19:25:56.450376 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.450378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.450644 (+ 266us) server_negotiation.cc:407] Connection header received
0809 19:25:56.450749 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.450750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.450766 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.450798 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:56.452045 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.452794 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.454108 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.454426 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.454503 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.455781 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:56.455783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:56.455783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:56.455791 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:56.457933 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.458371 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.458372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.458372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.458428 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.458648 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.458649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.458649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.458775 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:56.458896 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.459015 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.459103 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:25:56.459192 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:56.450560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40884 (local address 127.21.81.252:41405)
0809 19:25:56.450604 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:56.450606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:56.450742 (+ 136us) server_negotiation.cc:407] Connection header received
0809 19:25:56.450839 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:56.450840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:56.450851 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:56.450878 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:56.451857 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.452657 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.453844 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:56.454078 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:56.454155 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:56.455860 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:56.455862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:56.455862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:56.455868 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:56.457732 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.458147 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.458149 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.458149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.458230 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:56.458504 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:56.458506 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:56.458506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:56.458660 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:56.458744 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:56.458993 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 19:25:56.459088 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:25:57.427811 29689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:57.427465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.193:52619)
0809 19:25:57.427609 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:57.427705 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
W20260809 19:25:57.428262 29330 heartbeater.cc:644] Failed to heartbeat to 127.21.81.254:42869 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
I20260809 19:25:57.428329 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:57.427962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.194:56887)
0809 19:25:57.428204 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:57.428282 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":70,"threads_started":1}
W20260809 19:25:57.428540 29473 heartbeater.cc:644] Failed to heartbeat to 127.21.81.254:42869 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
I20260809 19:25:57.429427 29691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:57.429168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.195:47009)
0809 19:25:57.429314 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:57.429389 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
W20260809 19:25:57.429653 29621 heartbeater.cc:644] Failed to heartbeat to 127.21.81.254:42869 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
I20260809 19:25:57.932644 29694 raft_consensus.cc:482] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 62c195d9d60c46f08e956f8b6e3e88d6)
I20260809 19:25:57.932639 29692 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 62c195d9d60c46f08e956f8b6e3e88d6)
I20260809 19:25:57.932765 29694 raft_consensus.cc:504] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } }
I20260809 19:25:57.932802 29692 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } }
I20260809 19:25:57.933070 29694 leader_election.cc:288] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 62c195d9d60c46f08e956f8b6e3e88d6 (127.21.81.254:42869), 9720c6304069425a9b0f6fe82ac6c2cb (127.21.81.252:41405)
I20260809 19:25:57.933087 29692 leader_election.cc:288] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 62c195d9d60c46f08e956f8b6e3e88d6 (127.21.81.254:42869), b333cac93a8642d884e8e6b1d6ca05c8 (127.21.81.253:45301)
I20260809 19:25:57.933496 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:57.933241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:60288)
0809 19:25:57.933391 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:57.933451 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":53,"threads_started":1}
I20260809 19:25:57.933689 29698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:57.933255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:60298)
0809 19:25:57.933469 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:57.933525 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":44,"threads_started":1}
W20260809 19:25:57.933789 29123 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
W20260809 19:25:57.933873 28919 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
I20260809 19:25:57.934224 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:57.934131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:60306)
0809 19:25:57.934169 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:57.934199 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 19:25:57.934319 29123 leader_election.cc:334] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 62c195d9d60c46f08e956f8b6e3e88d6 (127.21.81.254:42869): Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
I20260809 19:25:57.934413 29698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:57.934312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:60312)
0809 19:25:57.934357 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:57.934386 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 19:25:57.934512 28919 leader_election.cc:334] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 62c195d9d60c46f08e956f8b6e3e88d6 (127.21.81.254:42869): Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
I20260809 19:25:57.937014 29699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:57.933271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:45301 (local address 127.0.0.1:46060)
0809 19:25:57.933402 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:57.933416 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:57.933477 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:57.933775 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:57.933776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:57.933779 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:57.933843 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:57.933847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:57.934834 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:57.934835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:57.936022 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:57.936025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:57.936752 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:57.936752 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:57.936834 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:57.936838 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:57.936885 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:25:57.936924 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":20,"thread_start_us":50,"threads_started":1}
I20260809 19:25:57.937027 29700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:57.933319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46060 (local address 127.21.81.253:45301)
0809 19:25:57.933570 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:57.933590 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:57.933598 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:57.933626 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:57.933627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:57.933639 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:57.933716 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:57.933926 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:57.934725 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:57.936126 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:57.936660 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:57.936740 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:57.936752 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:57.936908 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 19:25:57.936959 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":44,"threads_started":1}
I20260809 19:25:57.937242 29702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:57.933662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40898 (local address 127.21.81.252:41405)
0809 19:25:57.933812 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:57.933814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:57.933821 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:25:57.933838 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:57.933839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:57.933848 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:57.934033 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:57.934063 (+ 30us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:57.934991 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:57.936341 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:57.936916 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:57.936981 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:57.936991 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:57.937150 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 19:25:57.937187 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":134,"thread_start_us":57,"threads_started":1}
I20260809 19:25:57.937252 29701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:57.933409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:41405 (local address 127.0.0.1:40898)
0809 19:25:57.933671 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:57.933680 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:57.933723 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:57.933940 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:57.933941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:57.933942 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:57.933976 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:57.933977 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:57.935093 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:57.935094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:57.936256 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:57.936259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:57.937000 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:57.937001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:57.937071 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:57.937076 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:25:57.937142 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 19:25:57.937186 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":34,"threads_started":1}
I20260809 19:25:57.937515 29149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" is_pre_election: true
I20260809 19:25:57.937675 29149 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b333cac93a8642d884e8e6b1d6ca05c8 in term 1.
I20260809 19:25:57.937263 28946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" is_pre_election: true
I20260809 19:25:57.937747 28946 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9720c6304069425a9b0f6fe82ac6c2cb in term 1.
I20260809 19:25:57.937855 28919 leader_election.cc:302] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9720c6304069425a9b0f6fe82ac6c2cb, b333cac93a8642d884e8e6b1d6ca05c8; no voters: 62c195d9d60c46f08e956f8b6e3e88d6
I20260809 19:25:57.937897 29124 leader_election.cc:302] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9720c6304069425a9b0f6fe82ac6c2cb, b333cac93a8642d884e8e6b1d6ca05c8; no voters: 62c195d9d60c46f08e956f8b6e3e88d6
I20260809 19:25:57.937999 29694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 19:25:57.938017 29692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 19:25:57.938045 29694 raft_consensus.cc:482] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 62c195d9d60c46f08e956f8b6e3e88d6)
I20260809 19:25:57.938073 29694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:25:57.938059 29692 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 62c195d9d60c46f08e956f8b6e3e88d6)
I20260809 19:25:57.938107 29692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:25:57.938809 29694 raft_consensus.cc:504] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } }
I20260809 19:25:57.938850 29692 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } }
I20260809 19:25:57.938948 29694 leader_election.cc:288] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [CANDIDATE]: Term 2 election: Requested vote from peers 62c195d9d60c46f08e956f8b6e3e88d6 (127.21.81.254:42869), 9720c6304069425a9b0f6fe82ac6c2cb (127.21.81.252:41405)
I20260809 19:25:57.938977 29692 leader_election.cc:288] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [CANDIDATE]: Term 2 election: Requested vote from peers 62c195d9d60c46f08e956f8b6e3e88d6 (127.21.81.254:42869), b333cac93a8642d884e8e6b1d6ca05c8 (127.21.81.253:45301)
I20260809 19:25:57.939160 29701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:57.939031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:60320)
0809 19:25:57.939082 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:57.939133 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 19:25:57.939196 29699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:57.939093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:60322)
0809 19:25:57.939143 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:57.939171 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 19:25:57.939193 29149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9720c6304069425a9b0f6fe82ac6c2cb"
I20260809 19:25:57.939297 29149 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b333cac93a8642d884e8e6b1d6ca05c8 in current term 2: Already voted for candidate 9720c6304069425a9b0f6fe82ac6c2cb in this term.
I20260809 19:25:57.939306 28946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b333cac93a8642d884e8e6b1d6ca05c8"
I20260809 19:25:57.939383 28946 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9720c6304069425a9b0f6fe82ac6c2cb in current term 2: Already voted for candidate b333cac93a8642d884e8e6b1d6ca05c8 in this term.
I20260809 19:25:57.939531 29699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:57.939448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:60324)
0809 19:25:57.939481 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:57.939512 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 19:25:57.939628 29701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:57.939509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:60328)
0809 19:25:57.939566 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:57.939596 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 19:25:57.939703 28919 leader_election.cc:334] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 62c195d9d60c46f08e956f8b6e3e88d6 (127.21.81.254:42869): Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
W20260809 19:25:57.939706 29123 leader_election.cc:334] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 62c195d9d60c46f08e956f8b6e3e88d6 (127.21.81.254:42869): Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
I20260809 19:25:57.939747 28919 leader_election.cc:302] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b333cac93a8642d884e8e6b1d6ca05c8; no voters: 62c195d9d60c46f08e956f8b6e3e88d6, 9720c6304069425a9b0f6fe82ac6c2cb
I20260809 19:25:57.939747 29123 leader_election.cc:302] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9720c6304069425a9b0f6fe82ac6c2cb; no voters: 62c195d9d60c46f08e956f8b6e3e88d6, b333cac93a8642d884e8e6b1d6ca05c8
I20260809 19:25:57.939842 29694 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 19:25:57.939913 29692 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 19:25:58.428951 29705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:58.428680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.193:50021)
0809 19:25:58.428843 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:58.428888 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":67,"threads_started":1}
I20260809 19:25:58.429045 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:58.428824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.194:47841)
0809 19:25:58.428942 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:58.428992 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260809 19:25:58.430119 29707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:58.429911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.195:50751)
0809 19:25:58.430052 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:58.430084 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260809 19:25:59.429657 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.429376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.193:42709)
0809 19:25:59.429518 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.429597 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260809 19:25:59.429992 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.429448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.194:45721)
0809 19:25:59.429732 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.429920 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":47,"threads_started":1}
I20260809 19:25:59.430693 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.430476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.195:35531)
0809 19:25:59.430611 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.430658 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260809 19:25:59.505795 29711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:25:59.505893 29711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } }
I20260809 19:25:59.506135 29711 leader_election.cc:288] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 62c195d9d60c46f08e956f8b6e3e88d6 (127.21.81.254:42869), b333cac93a8642d884e8e6b1d6ca05c8 (127.21.81.253:45301)
I20260809 19:25:59.506436 28946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" is_pre_election: true
I20260809 19:25:59.506549 28946 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9720c6304069425a9b0f6fe82ac6c2cb in term 2.
I20260809 19:25:59.506549 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.506309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:60560)
0809 19:25:59.506462 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.506508 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":105,"threads_started":1}
I20260809 19:25:59.506759 29124 leader_election.cc:302] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9720c6304069425a9b0f6fe82ac6c2cb, b333cac93a8642d884e8e6b1d6ca05c8; no voters:
I20260809 19:25:59.506852 29711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 19:25:59.506897 29711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:25:59.506928 29711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 2 FOLLOWER]: Advancing to term 3
I20260809 19:25:59.507057 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.506962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:60568)
0809 19:25:59.507005 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.507029 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 19:25:59.507115 29123 leader_election.cc:334] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 62c195d9d60c46f08e956f8b6e3e88d6 (127.21.81.254:42869): Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
I20260809 19:25:59.507740 29711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } }
I20260809 19:25:59.507879 29711 leader_election.cc:288] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [CANDIDATE]: Term 3 election: Requested vote from peers 62c195d9d60c46f08e956f8b6e3e88d6 (127.21.81.254:42869), b333cac93a8642d884e8e6b1d6ca05c8 (127.21.81.253:45301)
I20260809 19:25:59.508051 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.507946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:60574)
0809 19:25:59.508001 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.508024 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 19:25:59.508095 28946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b333cac93a8642d884e8e6b1d6ca05c8"
I20260809 19:25:59.508189 28946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 2 FOLLOWER]: Advancing to term 3
I20260809 19:25:59.508381 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.508295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:60586)
0809 19:25:59.508338 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.508361 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 19:25:59.508455 29123 leader_election.cc:334] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 62c195d9d60c46f08e956f8b6e3e88d6 (127.21.81.254:42869): Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
I20260809 19:25:59.508946 28946 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9720c6304069425a9b0f6fe82ac6c2cb in term 3.
I20260809 19:25:59.509104 29124 leader_election.cc:302] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9720c6304069425a9b0f6fe82ac6c2cb, b333cac93a8642d884e8e6b1d6ca05c8; no voters: 62c195d9d60c46f08e956f8b6e3e88d6
I20260809 19:25:59.509204 29711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 3 FOLLOWER]: Leader election won for term 3
I20260809 19:25:59.509330 29711 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [term 3 LEADER]: Becoming Leader. State: Replica: 9720c6304069425a9b0f6fe82ac6c2cb, State: Running, Role: LEADER
I20260809 19:25:59.509423 29711 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } }
I20260809 19:25:59.509824 29714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9720c6304069425a9b0f6fe82ac6c2cb. Latest consensus state: current_term: 3 leader_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } } }
I20260809 19:25:59.509927 29714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:59.510114 29716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:59.510511 29716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:59.510571 29716 catalog_manager.cc:1259] Loaded cluster ID: f65849561df347f09a62f5020a8d2aa1
I20260809 19:25:59.510587 29716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:59.510792 29716 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:59.510874 29716 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: Loaded TSK: 0
I20260809 19:25:59.511021 29716 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:25:59.547583 29139 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:25:59.549355 29139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:25:59.550631 28946 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [term 3 FOLLOWER]: Refusing update from remote peer 9720c6304069425a9b0f6fe82ac6c2cb: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260809 19:25:59.550662 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.550504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:60608)
0809 19:25:59.550565 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.550607 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 19:25:59.550885 29711 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 19:25:59.551118 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.550996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.0.0.1:60624)
0809 19:25:59.551049 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.551087 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 19:25:59.551223 29123 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb -> Peer 62c195d9d60c46f08e956f8b6e3e88d6 (127.21.81.254:42869): Couldn't send request to peer 62c195d9d60c46f08e956f8b6e3e88d6. Status: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 19:25:59.551735 29719 sys_catalog.cc:455] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9720c6304069425a9b0f6fe82ac6c2cb. Latest consensus state: current_term: 3 leader_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } } }
I20260809 19:25:59.551864 29719 sys_catalog.cc:458] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:59.552414 29714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } } }
I20260809 19:25:59.552510 29714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:59.552654 29719 sys_catalog.cc:455] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "62c195d9d60c46f08e956f8b6e3e88d6" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42869 } } peers { permanent_uuid: "b333cac93a8642d884e8e6b1d6ca05c8" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 45301 } attrs { promote: false } } peers { permanent_uuid: "9720c6304069425a9b0f6fe82ac6c2cb" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 41405 } attrs { promote: false } } }
I20260809 19:25:59.552763 29719 sys_catalog.cc:458] T 00000000000000000000000000000000 P b333cac93a8642d884e8e6b1d6ca05c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:59.557797 29729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.554719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48512 (local address 127.21.81.194:41395)
0809 19:25:59.554836 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:59.554838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:59.554883 (+ 45us) server_negotiation.cc:407] Connection header received
0809 19:25:59.554952 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:59.554953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:59.554964 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:59.555158 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:59.555197 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:59.555913 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:59.556842 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:59.557451 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:59.557518 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:59.557530 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:59.557675 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 19:25:59.557732 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260809 19:25:59.557788 29728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.554488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:41395 (local address 127.0.0.1:48512)
0809 19:25:59.554839 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.554847 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:59.554890 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:59.555062 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:59.555063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:59.555065 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:59.555097 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:59.555098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:59.556013 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:59.556013 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:59.556752 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:59.556753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:59.557524 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:59.557525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:59.557605 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:59.557610 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:25:59.557666 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 19:25:59.557711 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":98,"threads_started":1}
I20260809 19:25:59.558094 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.554398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:44841 (local address 127.0.0.1:51752)
0809 19:25:59.554625 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.554637 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:59.554705 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:59.554906 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:59.554907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:59.554909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:59.554973 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:59.554976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:59.555951 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:59.555951 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:59.557121 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:59.557124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:59.557885 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:59.557886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:59.557953 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:59.557957 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:59.557985 (+ 28us) client_negotiation.cc:240] Negotiation successful
0809 19:25:59.558025 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":31}
I20260809 19:25:59.558432 29726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.554496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51752 (local address 127.21.81.195:44841)
0809 19:25:59.554643 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:59.554645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:59.554732 (+ 87us) server_negotiation.cc:407] Connection header received
0809 19:25:59.554775 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:59.554776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:59.554789 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:59.555044 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:59.555087 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:59.555882 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:59.557204 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:59.557787 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:59.558283 (+ 496us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:59.558300 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:59.558320 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 19:25:59.558368 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260809 19:25:59.558611 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.554485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:44385 (local address 127.0.0.1:40772)
0809 19:25:59.554711 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.554721 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:59.554768 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:59.555141 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:59.555142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:59.555144 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:25:59.555198 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:59.555199 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:59.555887 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:59.555888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:59.557131 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:59.557133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:59.558336 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:59.558337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:59.558416 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:59.558420 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:25:59.558472 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 19:25:59.558534 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":25,"thread_start_us":125,"threads_started":1}
I20260809 19:25:59.558637 29405 tablet_service.cc:1467] Processing CreateTablet for tablet 292e7c8cf1f94956990324c3e1c343ec (DEFAULT_TABLE table=test-table [id=9eae66763562472ba1dac13de31aff78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:59.558787 29405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292e7c8cf1f94956990324c3e1c343ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:59.558871 29731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.554862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40772 (local address 127.21.81.193:44385)
0809 19:25:59.554994 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:59.554996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:59.555004 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:59.555031 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:59.555032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:59.555044 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:59.555268 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:25:59.555316 (+ 48us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:59.555804 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:59.557633 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:59.558247 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:59.558311 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:59.558323 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:59.558736 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 19:25:59.558787 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":85,"threads_started":1}
I20260809 19:25:59.559103 29259 tablet_service.cc:1467] Processing CreateTablet for tablet 292e7c8cf1f94956990324c3e1c343ec (DEFAULT_TABLE table=test-table [id=9eae66763562472ba1dac13de31aff78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:59.559237 29259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292e7c8cf1f94956990324c3e1c343ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:59.560247 29732 tablet_bootstrap.cc:492] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d: Bootstrap starting.
I20260809 19:25:59.560288 29546 tablet_service.cc:1467] Processing CreateTablet for tablet 292e7c8cf1f94956990324c3e1c343ec (DEFAULT_TABLE table=test-table [id=9eae66763562472ba1dac13de31aff78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:25:59.560406 29546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292e7c8cf1f94956990324c3e1c343ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:59.560711 29732 tablet_bootstrap.cc:654] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:59.561266 29732 tablet_bootstrap.cc:492] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d: No bootstrap required, opened a new log
I20260809 19:25:59.561316 29732 ts_tablet_manager.cc:1397] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:59.561484 29732 raft_consensus.cc:348] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } } peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } }
I20260809 19:25:59.561549 29732 raft_consensus.cc:374] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:59.561568 29732 raft_consensus.cc:729] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51eb4d4e6874518a2d659a44cfe7d4d, State: Initialized, Role: FOLLOWER
I20260809 19:25:59.561663 29732 consensus_queue.cc:260] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } } peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } }
I20260809 19:25:59.561739 29733 tablet_bootstrap.cc:492] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135: Bootstrap starting.
I20260809 19:25:59.562106 29733 tablet_bootstrap.cc:654] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:59.562202 29732 ts_tablet_manager.cc:1428] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:25:59.562325 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.562185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.194:60575)
0809 19:25:59.562257 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.562292 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 19:25:59.562698 29472 heartbeater.cc:503] Master 127.21.81.252:41405 was elected leader, sending a full tablet report...
I20260809 19:25:59.562726 29736 tablet_bootstrap.cc:492] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2: Bootstrap starting.
I20260809 19:25:59.562764 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.562598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.194:36657)
0809 19:25:59.562650 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.562683 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 19:25:59.563374 29733 tablet_bootstrap.cc:492] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135: No bootstrap required, opened a new log
I20260809 19:25:59.563439 29733 ts_tablet_manager.cc:1397] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:25:59.563530 29736 tablet_bootstrap.cc:654] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:59.563624 29733 raft_consensus.cc:348] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } } peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } }
I20260809 19:25:59.563719 29733 raft_consensus.cc:374] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:59.563779 29733 raft_consensus.cc:729] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c48d2cd2e04bc9a3c56487fea6b135, State: Initialized, Role: FOLLOWER
I20260809 19:25:59.563841 29733 consensus_queue.cc:260] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } } peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } }
I20260809 19:25:59.564019 29733 ts_tablet_manager.cc:1428] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:59.564157 29736 tablet_bootstrap.cc:492] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2: No bootstrap required, opened a new log
I20260809 19:25:59.564230 29736 ts_tablet_manager.cc:1397] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:25:59.564344 29736 raft_consensus.cc:348] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } } peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } }
I20260809 19:25:59.564388 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.564260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.193:40051)
0809 19:25:59.564333 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.564360 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 19:25:59.564417 29736 raft_consensus.cc:374] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:59.564440 29736 raft_consensus.cc:729] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d4d3cc2b034776ac860df5fa8c1cf2, State: Initialized, Role: FOLLOWER
I20260809 19:25:59.564504 29736 consensus_queue.cc:260] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } } peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } }
I20260809 19:25:59.564643 29329 heartbeater.cc:503] Master 127.21.81.252:41405 was elected leader, sending a full tablet report...
I20260809 19:25:59.564700 29736 ts_tablet_manager.cc:1428] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:59.564850 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.564572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.193:40655)
0809 19:25:59.564632 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.564655 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 19:25:59.565090 29620 heartbeater.cc:503] Master 127.21.81.252:41405 was elected leader, sending a full tablet report...
I20260809 19:25:59.565140 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.565014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.195:49255)
0809 19:25:59.565090 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.565117 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 19:25:59.565493 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.565371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.195:59911)
0809 19:25:59.565441 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.565471 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 19:25:59.683331 29332 tablet.cc:2369] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:25:59.738540 29623 tablet.cc:2369] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:25:59.917675 29740 raft_consensus.cc:482] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:25:59.917781 29740 raft_consensus.cc:504] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } } peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } }
I20260809 19:25:59.917956 29740 leader_election.cc:288] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10c48d2cd2e04bc9a3c56487fea6b135 (127.21.81.193:44385), d51eb4d4e6874518a2d659a44cfe7d4d (127.21.81.194:41395)
I20260809 19:25:59.918234 29425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292e7c8cf1f94956990324c3e1c343ec" candidate_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" is_pre_election: true
I20260809 19:25:59.918215 29280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292e7c8cf1f94956990324c3e1c343ec" candidate_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" is_pre_election: true
I20260809 19:25:59.918340 29280 raft_consensus.cc:2454] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00d4d3cc2b034776ac860df5fa8c1cf2 in term 0.
I20260809 19:25:59.918340 29425 raft_consensus.cc:2454] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00d4d3cc2b034776ac860df5fa8c1cf2 in term 0.
I20260809 19:25:59.918511 29498 leader_election.cc:302] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d4d3cc2b034776ac860df5fa8c1cf2, d51eb4d4e6874518a2d659a44cfe7d4d; no voters:
I20260809 19:25:59.918648 29740 raft_consensus.cc:2781] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:25:59.918697 29740 raft_consensus.cc:482] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:25:59.918725 29740 raft_consensus.cc:3037] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:59.919483 29740 raft_consensus.cc:504] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } } peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } }
I20260809 19:25:59.919632 29740 leader_election.cc:288] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [CANDIDATE]: Term 1 election: Requested vote from peers 10c48d2cd2e04bc9a3c56487fea6b135 (127.21.81.193:44385), d51eb4d4e6874518a2d659a44cfe7d4d (127.21.81.194:41395)
I20260809 19:25:59.919816 29280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292e7c8cf1f94956990324c3e1c343ec" candidate_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10c48d2cd2e04bc9a3c56487fea6b135"
I20260809 19:25:59.919823 29425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292e7c8cf1f94956990324c3e1c343ec" candidate_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d"
I20260809 19:25:59.919909 29425 raft_consensus.cc:3037] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:59.919912 29280 raft_consensus.cc:3037] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:59.920521 29280 raft_consensus.cc:2454] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d4d3cc2b034776ac860df5fa8c1cf2 in term 1.
I20260809 19:25:59.920542 29425 raft_consensus.cc:2454] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d4d3cc2b034776ac860df5fa8c1cf2 in term 1.
I20260809 19:25:59.920725 29498 leader_election.cc:302] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d4d3cc2b034776ac860df5fa8c1cf2, d51eb4d4e6874518a2d659a44cfe7d4d; no voters:
I20260809 19:25:59.920852 29740 raft_consensus.cc:2781] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:59.921063 29740 raft_consensus.cc:686] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 1 LEADER]: Becoming Leader. State: Replica: 00d4d3cc2b034776ac860df5fa8c1cf2, State: Running, Role: LEADER
I20260809 19:25:59.921149 29740 consensus_queue.cc:237] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } } peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } }
I20260809 19:25:59.921504 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.921367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.195:33103)
0809 19:25:59.921427 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.921470 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 19:25:59.921888 29138 catalog_manager.cc:5495] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00d4d3cc2b034776ac860df5fa8c1cf2 (127.21.81.195). New cstate: current_term: 1 leader_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d4d3cc2b034776ac860df5fa8c1cf2" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 } health_report { overall_health: UNKNOWN } } }
I20260809 19:25:59.997538 29726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.995273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51756 (local address 127.21.81.195:44841)
0809 19:25:59.995338 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:59.995341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:59.995351 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:25:59.995380 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:59.995381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:59.995391 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:59.995460 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:59.995598 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:59.996083 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:59.996836 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:59.997035 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:59.997108 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:59.997271 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:59.997307 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:59.997361 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:59.997421 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 19:25:59.997445 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:25:59.998548 29280 raft_consensus.cc:1261] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135 [term 1 FOLLOWER]: Refusing update from remote peer 00d4d3cc2b034776ac860df5fa8c1cf2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:59.998543 29425 raft_consensus.cc:1261] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d [term 1 FOLLOWER]: Refusing update from remote peer 00d4d3cc2b034776ac860df5fa8c1cf2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:59.998797 29741 consensus_queue.cc:1035] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10c48d2cd2e04bc9a3c56487fea6b135" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:59.998907 29740 consensus_queue.cc:1035] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51eb4d4e6874518a2d659a44cfe7d4d" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 41395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:59.999720 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.999571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.193:32949)
0809 19:25:59.999646 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.999685 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 19:25:59.999960 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:59.999776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.194:51903)
0809 19:25:59.999890 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:59.999922 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 19:26:00.000829 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:00.000539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.195:56269)
0809 19:26:00.000757 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:00.000795 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197}
I20260809 19:26:00.001349 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:00.001248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.194:46273)
0809 19:26:00.001289 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:00.001322 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 19:26:00.001523 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:00.001388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.195:58861)
0809 19:26:00.001464 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:00.001491 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 19:26:00.001724 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:00.001259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42869 (local address 127.21.81.193:47125)
0809 19:26:00.001669 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:00.001700 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391}
I20260809 19:26:00.003539 29138 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:26:00.003628 29138 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:26:00.004956 29138 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9720c6304069425a9b0f6fe82ac6c2cb: Sending DeleteTablet for 3 replicas of tablet 292e7c8cf1f94956990324c3e1c343ec
I20260809 19:26:00.005230 29546 tablet_service.cc:1514] Processing DeleteTablet for tablet 292e7c8cf1f94956990324c3e1c343ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:00 UTC) from {username='oryx', principal='oryx/127.21.81.252@KRBTEST.COM'} at 127.0.0.1:51752
I20260809 19:26:00.005342 29259 tablet_service.cc:1514] Processing DeleteTablet for tablet 292e7c8cf1f94956990324c3e1c343ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:00 UTC) from {username='oryx', principal='oryx/127.21.81.252@KRBTEST.COM'} at 127.0.0.1:40772
I20260809 19:26:00.005474 29405 tablet_service.cc:1514] Processing DeleteTablet for tablet 292e7c8cf1f94956990324c3e1c343ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:00 UTC) from {username='oryx', principal='oryx/127.21.81.252@KRBTEST.COM'} at 127.0.0.1:48512
I20260809 19:26:00.005535 29754 tablet_replica.cc:330] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135: stopping tablet replica
I20260809 19:26:00.005609 29753 tablet_replica.cc:330] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2: stopping tablet replica
I20260809 19:26:00.005653 29754 raft_consensus.cc:2229] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:26:00.005707 29755 tablet_replica.cc:330] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d: stopping tablet replica
I20260809 19:26:00.005734 29754 raft_consensus.cc:2258] T 292e7c8cf1f94956990324c3e1c343ec P 10c48d2cd2e04bc9a3c56487fea6b135 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:00.005776 29755 raft_consensus.cc:2229] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:26:00.005813 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 29100
I20260809 19:26:00.005841 29755 raft_consensus.cc:2258] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:00.006947 29755 ts_tablet_manager.cc:1905] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:26:00.008423 29755 ts_tablet_manager.cc:1918] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:26:00.008467 29755 log.cc:1196] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/292e7c8cf1f94956990324c3e1c343ec
I20260809 19:26:00.008646 29755 ts_tablet_manager.cc:1939] T 292e7c8cf1f94956990324c3e1c343ec P d51eb4d4e6874518a2d659a44cfe7d4d: Deleting consensus metadata
I20260809 19:26:00.009007 29124 catalog_manager.cc:4841] TS d51eb4d4e6874518a2d659a44cfe7d4d (127.21.81.194:41395): tablet 292e7c8cf1f94956990324c3e1c343ec (table test-table [id=9eae66763562472ba1dac13de31aff78]) successfully deleted
I20260809 19:26:00.011220 29753 raft_consensus.cc:2229] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:26:00.011322 29753 raft_consensus.cc:2258] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:00.011629 29753 ts_tablet_manager.cc:1905] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:26:00.013934 29753 ts_tablet_manager.cc:1918] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:26:00.013989 29753 log.cc:1196] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/292e7c8cf1f94956990324c3e1c343ec
I20260809 19:26:00.014223 29753 ts_tablet_manager.cc:1939] T 292e7c8cf1f94956990324c3e1c343ec P 00d4d3cc2b034776ac860df5fa8c1cf2: Deleting consensus metadata
I20260809 19:26:00.014914 29124 catalog_manager.cc:4841] TS 00d4d3cc2b034776ac860df5fa8c1cf2 (127.21.81.195:44841): tablet 292e7c8cf1f94956990324c3e1c343ec (table test-table [id=9eae66763562472ba1dac13de31aff78]) successfully deleted
I20260809 19:26:00.019608 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:00.019473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:44385 (local address 127.0.0.1:40786)
0809 19:26:00.019535 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:00.019569 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:44385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 19:26:00.019706 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 29337
W20260809 19:26:00.019786 29123 catalog_manager.cc:4584] TS 10c48d2cd2e04bc9a3c56487fea6b135 (127.21.81.193:44385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 292e7c8cf1f94956990324c3e1c343ec: Network error: Client connection negotiation failed: client connection to 127.21.81.193:44385: connect: Connection refused (error 111)
I20260809 19:26:00.026613 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 29479
I20260809 19:26:00.033860 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 28787
I20260809 19:26:00.040021 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 28995
2026-08-09T19:26:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6706 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:26:00 dist-test-slave-njxd krb5kdc[29758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:26:00 dist-test-slave-njxd krb5kdc[29758](info): set up 2 sockets
Aug 09 19:26:00 dist-test-slave-njxd krb5kdc[29758](info): commencing operation
krb5kdc: starting...
W20260809 19:26:01.130219 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:26:01 dist-test-slave-njxd krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:26:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:26:01Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.254.keytab.
Entry for principal oryx/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.254.keytab.
I20260809 19:26:01.253914 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:34111
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.254.keytab
--principal=oryx/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:34111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:26:01.335795 29773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:01.335999 29773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:01.336022 29773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:01.337159 29773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:26:01.337194 29773 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 19:26:01.337208 29773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:01.337220 29773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:01.337232 29773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:26:01.337244 29773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:26:01.338770 29773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:34111
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.254.keytab
--principal=oryx/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:34111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:01.338963 29773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:01.339138 29773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:01.343907 29778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:01.343920 29781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:01.343924 29779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:01.344049 29773 server_base.cc:1034] running on GCE node
I20260809 19:26:01.344619 29773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:01.344870 29773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:01.346007 29773 hybrid_clock.cc:648] HybridClock initialized: now 1786303561345989 us; error 39 us; skew 500 ppm
Aug 09 19:26:01 dist-test-slave-njxd krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303561, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:01.348479 29773 init.cc:377] Logged in from keytab as oryx/127.21.81.254@KRBTEST.COM (short username oryx)
I20260809 19:26:01.348896 29773 webserver.cc:460] Webserver started at http://127.21.81.254:35315/ using document root <none> and password file <none>
I20260809 19:26:01.349110 29773 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:01.349157 29773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:01.349331 29773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:01.350441 29773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "614bb27e14f14f1fbe5f3c0b0b43b91a"
format_stamp: "Formatted at 2026-08-09 19:26:01 on dist-test-slave-njxd"
server_key: "c0441d4070a070f9cf59ca173d36f86f"
server_key_iv: "1482d016bbcf542d7e8fc2d30b41814c"
server_key_version: "encryptionkey@0"
I20260809 19:26:01.350814 29773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "614bb27e14f14f1fbe5f3c0b0b43b91a"
format_stamp: "Formatted at 2026-08-09 19:26:01 on dist-test-slave-njxd"
server_key: "c0441d4070a070f9cf59ca173d36f86f"
server_key_iv: "1482d016bbcf542d7e8fc2d30b41814c"
server_key_version: "encryptionkey@0"
I20260809 19:26:01.352164 29773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:26:01.352905 29788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:01.353076 29773 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:26:01.353188 29773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "614bb27e14f14f1fbe5f3c0b0b43b91a"
format_stamp: "Formatted at 2026-08-09 19:26:01 on dist-test-slave-njxd"
server_key: "c0441d4070a070f9cf59ca173d36f86f"
server_key_iv: "1482d016bbcf542d7e8fc2d30b41814c"
server_key_version: "encryptionkey@0"
I20260809 19:26:01.353272 29773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:01.370353 29773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:01.372184 29773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:01.372347 29773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:01.377214 29773 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:34111
I20260809 19:26:01.377275 29853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:34111 every 8 connection(s)
I20260809 19:26:01.377676 29773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:26:01.378427 29855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:01.378799 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 29773
I20260809 19:26:01.378911 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:26:01.379173 21831 external_mini_cluster.cc:1428] Setting key ea6e376a5a8a5ad3e573e03d171cd245
I20260809 19:26:01.381517 29855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a: Bootstrap starting.
I20260809 19:26:01.382265 29855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:01.382571 29855 log.cc:824] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:26:01.383236 29855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a: No bootstrap required, opened a new log
I20260809 19:26:01.385279 29855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "614bb27e14f14f1fbe5f3c0b0b43b91a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34111 } }
I20260809 19:26:01.385437 29855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:01.385499 29855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 614bb27e14f14f1fbe5f3c0b0b43b91a, State: Initialized, Role: FOLLOWER
I20260809 19:26:01.385701 29855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "614bb27e14f14f1fbe5f3c0b0b43b91a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34111 } }
I20260809 19:26:01.385800 29855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:26:01.385846 29855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:26:01.385896 29855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [term 0 FOLLOWER]: Advancing to term 1
Aug 09 19:26:01 dist-test-slave-njxd krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
I20260809 19:26:01.386498 29855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "614bb27e14f14f1fbe5f3c0b0b43b91a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34111 } }
I20260809 19:26:01.386627 29855 leader_election.cc:302] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 614bb27e14f14f1fbe5f3c0b0b43b91a; no voters:
I20260809 19:26:01.386806 29855 leader_election.cc:288] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:26:01.386849 29860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:26:01.387022 29860 raft_consensus.cc:686] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [term 1 LEADER]: Becoming Leader. State: Replica: 614bb27e14f14f1fbe5f3c0b0b43b91a, State: Running, Role: LEADER
I20260809 19:26:01.387084 29855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:26:01.387168 29860 consensus_queue.cc:237] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "614bb27e14f14f1fbe5f3c0b0b43b91a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34111 } }
I20260809 19:26:01.387498 29861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "614bb27e14f14f1fbe5f3c0b0b43b91a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "614bb27e14f14f1fbe5f3c0b0b43b91a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34111 } } }
I20260809 19:26:01.387609 29861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [sys.catalog]: This master's current role is: LEADER
I20260809 19:26:01.387487 29860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 614bb27e14f14f1fbe5f3c0b0b43b91a. Latest consensus state: current_term: 1 leader_uuid: "614bb27e14f14f1fbe5f3c0b0b43b91a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "614bb27e14f14f1fbe5f3c0b0b43b91a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34111 } } }
I20260809 19:26:01.387869 29860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a [sys.catalog]: This master's current role is: LEADER
I20260809 19:26:01.387990 29867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:26:01.388500 29867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:26:01.390306 29878 mvcc.cc:205] Tried to move back new op lower bound from 7316699387451146240 to 7316699387442241536. Current Snapshot: MvccSnapshot[applied={T|T < 7316699387451146240}]
I20260809 19:26:01.390331 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.379736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42962 (local address 127.21.81.254:34111)
0809 19:26:01.379953 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:01.379963 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:01.379998 (+ 35us) server_negotiation.cc:407] Connection header received
0809 19:26:01.380527 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:01.380536 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:01.380593 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:01.380764 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:01.381366 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.382759 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.383541 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.383927 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.384038 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.386541 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:01.386545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:01.386545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:01.386557 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:01.388702 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.389084 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.389086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.389086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.389180 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.389418 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.389419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.389419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.389711 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:01.389869 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:01.389986 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 19:26:01.390140 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":110,"threads_started":1}
I20260809 19:26:01.394014 29867 catalog_manager.cc:1347] Generated new cluster ID: 20b6eca5bc8640dfa6c8435c5ba31ad0
I20260809 19:26:01.394073 29867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:26:01.409471 29867 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:26:01.410223 29867 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:26:01.412938 29867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a: Generated new TSK 0
I20260809 19:26:01.413118 29867 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.193.keytab.
Entry for principal oryx/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.193.keytab.
I20260809 19:26:01.448032 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:34111
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.193.keytab
--principal=oryx/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:26:01.543792 29883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:01.543992 29883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:01.544016 29883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:01.545116 29883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:01.545150 29883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:01.545176 29883 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:26:01.545193 29883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:26:01.546715 29883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.193.keytab
--principal=oryx/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.81.254:34111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:01.546911 29883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:01.547092 29883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:01.551966 29889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:01.551973 29888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:01.552006 29891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:01.552294 29883 server_base.cc:1034] running on GCE node
I20260809 19:26:01.552507 29883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:01.552801 29883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:01.553959 29883 hybrid_clock.cc:648] HybridClock initialized: now 1786303561553950 us; error 28 us; skew 500 ppm
Aug 09 19:26:01 dist-test-slave-njxd krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303561, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:01.556492 29883 init.cc:377] Logged in from keytab as oryx/127.21.81.193@KRBTEST.COM (short username oryx)
I20260809 19:26:01.556995 29883 webserver.cc:460] Webserver started at http://127.21.81.193:39789/ using document root <none> and password file <none>
I20260809 19:26:01.557250 29883 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:01.557314 29883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:01.557457 29883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:01.558595 29883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "30a6da08dd544f4c8a4206925dd4fa46"
format_stamp: "Formatted at 2026-08-09 19:26:01 on dist-test-slave-njxd"
server_key: "79f0730a44fd66e948c793c6bbcbeb9c"
server_key_iv: "751032ae48180688cfe2a08979ac83c8"
server_key_version: "encryptionkey@0"
I20260809 19:26:01.558955 29883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "30a6da08dd544f4c8a4206925dd4fa46"
format_stamp: "Formatted at 2026-08-09 19:26:01 on dist-test-slave-njxd"
server_key: "79f0730a44fd66e948c793c6bbcbeb9c"
server_key_iv: "751032ae48180688cfe2a08979ac83c8"
server_key_version: "encryptionkey@0"
I20260809 19:26:01.560287 29883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:26:01.561002 29898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:01.561183 29883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:26:01.561283 29883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "30a6da08dd544f4c8a4206925dd4fa46"
format_stamp: "Formatted at 2026-08-09 19:26:01 on dist-test-slave-njxd"
server_key: "79f0730a44fd66e948c793c6bbcbeb9c"
server_key_iv: "751032ae48180688cfe2a08979ac83c8"
server_key_version: "encryptionkey@0"
I20260809 19:26:01.561360 29883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:01.574995 29883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:01.577090 29883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:01.577286 29883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:01.577988 29883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:01.578043 29883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:01.578078 29883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:01.578102 29883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:26:01 dist-test-slave-njxd krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303561, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
I20260809 19:26:01.585956 29883 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:42947
I20260809 19:26:01.586092 30021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:42947 every 8 connection(s)
I20260809 19:26:01.586385 29883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:26:01.589035 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.578501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:55435 (local address 127.21.81.254:34111)
0809 19:26:01.578586 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:01.578588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:01.578882 (+ 294us) server_negotiation.cc:407] Connection header received
0809 19:26:01.579638 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:01.579640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:01.579652 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:01.579699 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:01.580678 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.581197 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.582931 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.583213 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.583269 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.585997 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:01.585999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:01.585999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:01.586008 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:01.587630 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.588033 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.588034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.588035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.588119 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.588381 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.588382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.588383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.588506 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:01.588575 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:01.588879 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 19:26:01.588969 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 19:26:01.589184 29909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.578495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:34111 (local address 127.21.81.193:55435)
0809 19:26:01.578713 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:01.578771 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:01.579508 (+ 737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:01.579787 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:01.579792 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:01.580233 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:01.580560 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.580566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.581274 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.581275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.582815 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.582819 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.583293 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.583294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.583484 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.584073 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:01.584085 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:01.585896 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:01.587717 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:01.587725 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:01.587728 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:01.587956 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:01.588191 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:01.588192 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:01.588193 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:01.588310 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:01.588576 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:01.588585 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:01.588725 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 19:26:01.588836 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 19:26:01.588982 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":109,"threads_started":1}
I20260809 19:26:01.589699 30022 heartbeater.cc:347] Connected to a master server at 127.21.81.254:34111
I20260809 19:26:01.589783 30022 heartbeater.cc:464] Registering TS with master...
I20260809 19:26:01.589982 30022 heartbeater.cc:511] Master 127.21.81.254:34111 requested a full tablet report, sending...
I20260809 19:26:01.590513 29806 ts_manager.cc:194] Registered new tserver with Master: 30a6da08dd544f4c8a4206925dd4fa46 (127.21.81.193:42947)
I20260809 19:26:01.591423 29806 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:55435
I20260809 19:26:01.594676 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 29883
I20260809 19:26:01.594807 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:26:01.595037 21831 external_mini_cluster.cc:1428] Setting key 53da59206ed74cc362edb9ec91e1c1b6
I20260809 19:26:01.596494 30028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.590818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:34111 (local address 127.21.81.193:57243)
0809 19:26:01.590966 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:01.590977 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:01.591036 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:01.591154 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:01.591155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:01.591344 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:01.591413 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.591415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.592070 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.592071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.592654 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.592657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.592959 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.592960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.593007 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.593410 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:01.593414 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:01.593776 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:01.595461 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:01.595462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:01.595462 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:01.595640 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:01.595848 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:01.595849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:01.595849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:01.595914 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:01.596149 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:01.596150 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:01.596201 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 19:26:01.596280 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 19:26:01.596397 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260809 19:26:01.596490 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.590904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:57243 (local address 127.21.81.254:34111)
0809 19:26:01.590951 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:01.590953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:01.591021 (+ 68us) server_negotiation.cc:407] Connection header received
0809 19:26:01.591090 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:01.591091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:01.591099 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:01.591142 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:01.591503 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.591979 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.592724 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.592899 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.592948 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.593847 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:01.593848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:01.593848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:01.593853 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:01.595389 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.595713 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.595715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.595715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.595773 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.595973 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.595974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.595974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.596087 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:01.596158 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:01.596320 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 19:26:01.596394 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
WARNING: no policy specified for oryx/127.21.81.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.194.keytab.
Entry for principal oryx/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.194.keytab.
I20260809 19:26:01.620625 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:34111
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.194.keytab
--principal=oryx/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:26:01.703824 30031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:01.704017 30031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:01.704041 30031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:01.705114 30031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:01.705150 30031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:01.705170 30031 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:26:01.705188 30031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:26:01.706626 30031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.194.keytab
--principal=oryx/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.81.254:34111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:01.706804 30031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:01.706969 30031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:01.711860 30036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:01.711865 30039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:01.712029 30031 server_base.cc:1034] running on GCE node
W20260809 19:26:01.711951 30037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:01.712289 30031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:01.712553 30031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:01.713726 30031 hybrid_clock.cc:648] HybridClock initialized: now 1786303561713724 us; error 41 us; skew 500 ppm
Aug 09 19:26:01 dist-test-slave-njxd krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303561, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:01.716265 30031 init.cc:377] Logged in from keytab as oryx/127.21.81.194@KRBTEST.COM (short username oryx)
I20260809 19:26:01.716727 30031 webserver.cc:460] Webserver started at http://127.21.81.194:45125/ using document root <none> and password file <none>
I20260809 19:26:01.716948 30031 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:01.717012 30031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:01.717145 30031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:01.718264 30031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "24014678d6a345abaf05bbdb3d0003ca"
format_stamp: "Formatted at 2026-08-09 19:26:01 on dist-test-slave-njxd"
server_key: "f13b3fa1129fde30b7e7c97a5bd9ea3c"
server_key_iv: "cac44c994908bb6d74fa43f304ce1b64"
server_key_version: "encryptionkey@0"
I20260809 19:26:01.718619 30031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "24014678d6a345abaf05bbdb3d0003ca"
format_stamp: "Formatted at 2026-08-09 19:26:01 on dist-test-slave-njxd"
server_key: "f13b3fa1129fde30b7e7c97a5bd9ea3c"
server_key_iv: "cac44c994908bb6d74fa43f304ce1b64"
server_key_version: "encryptionkey@0"
I20260809 19:26:01.719928 30031 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:26:01.720707 30046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:01.720870 30031 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:26:01.720963 30031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "24014678d6a345abaf05bbdb3d0003ca"
format_stamp: "Formatted at 2026-08-09 19:26:01 on dist-test-slave-njxd"
server_key: "f13b3fa1129fde30b7e7c97a5bd9ea3c"
server_key_iv: "cac44c994908bb6d74fa43f304ce1b64"
server_key_version: "encryptionkey@0"
I20260809 19:26:01.721037 30031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:01.748083 30031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:01.750339 30031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:01.750510 30031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:01.751129 30031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:01.751186 30031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:01.751221 30031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:01.751267 30031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:26:01 dist-test-slave-njxd krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303561, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
I20260809 19:26:01.759222 30031 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:38535
I20260809 19:26:01.759598 30031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:26:01.759697 30169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:38535 every 8 connection(s)
I20260809 19:26:01.762014 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.751537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:34299 (local address 127.21.81.254:34111)
0809 19:26:01.751607 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:01.751609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:01.751919 (+ 310us) server_negotiation.cc:407] Connection header received
0809 19:26:01.752688 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:01.752689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:01.752701 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:01.752787 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:01.753896 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.754490 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.755701 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.756069 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.756156 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.758705 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:01.758707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:01.758708 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:01.758716 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:01.760471 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.760962 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.760964 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.760964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.761033 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.761297 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.761298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.761299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.761427 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:01.761497 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:01.761829 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 19:26:01.761920 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:26:01.762215 30057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.751533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:34111 (local address 127.21.81.194:34299)
0809 19:26:01.751748 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:01.751807 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:01.752570 (+ 763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:01.752880 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:01.752884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:01.753353 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:01.753728 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.753735 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.754606 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.754607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.755554 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.755557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.756181 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.756182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.756367 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.756999 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:01.757014 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:01.758603 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:01.760566 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:01.760575 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:01.760577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:01.760881 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:01.761099 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:01.761100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:01.761100 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:01.761223 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:01.761496 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:01.761505 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:01.761692 (+ 187us) client_negotiation.cc:769] Sending connection context
0809 19:26:01.761802 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 19:26:01.761974 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":109,"threads_started":1}
I20260809 19:26:01.762691 30170 heartbeater.cc:347] Connected to a master server at 127.21.81.254:34111
I20260809 19:26:01.762795 30170 heartbeater.cc:464] Registering TS with master...
I20260809 19:26:01.762981 30170 heartbeater.cc:511] Master 127.21.81.254:34111 requested a full tablet report, sending...
I20260809 19:26:01.763434 29806 ts_manager.cc:194] Registered new tserver with Master: 24014678d6a345abaf05bbdb3d0003ca (127.21.81.194:38535)
I20260809 19:26:01.763867 29806 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:34299
I20260809 19:26:01.766553 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 30031
I20260809 19:26:01.766683 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:26:01.766909 21831 external_mini_cluster.cc:1428] Setting key db11158b38b5f41a9dcde35071f3c016
I20260809 19:26:01.770149 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.764299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:38903 (local address 127.21.81.254:34111)
0809 19:26:01.764340 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:01.764342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:01.764502 (+ 160us) server_negotiation.cc:407] Connection header received
0809 19:26:01.764575 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:01.764576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:01.764585 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:01.764630 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:01.764966 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.765504 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.766247 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.766451 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.766522 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.767451 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:01.767453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:01.767453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:01.767460 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:01.768815 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.769249 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.769251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.769251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.769313 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.769525 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.769525 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.769526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.769731 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:01.769790 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:01.769998 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 19:26:01.770075 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:26:01.770190 30177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.764239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:34111 (local address 127.21.81.194:38903)
0809 19:26:01.764438 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:01.764448 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:01.764517 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:01.764635 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:01.764636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:01.764834 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:01.764887 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.764888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.765638 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.765639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.766148 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.766150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.766522 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.766522 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.766589 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.767046 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:01.767051 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:01.767347 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:01.768913 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:01.768914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:01.768915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:01.769162 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:01.769383 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:01.769384 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:01.769384 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:01.769460 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:01.769799 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:01.769800 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:01.769855 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 19:26:01.769934 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 19:26:01.770051 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":132,"threads_started":1}
WARNING: no policy specified for oryx/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.195.keytab.
Entry for principal oryx/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.195.keytab.
I20260809 19:26:01.792872 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:34111
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.195.keytab
--principal=oryx/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:26:01.874516 30180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:01.874728 30180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:01.874753 30180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:01.875835 30180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:01.875869 30180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:01.875890 30180 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:26:01.875906 30180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:26:01.877337 30180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.195.keytab
--principal=oryx/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.81.254:34111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:01.877518 30180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:01.877766 30180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:01.882601 30188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:01.882606 30185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:01.882596 30186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:01.882758 30180 server_base.cc:1034] running on GCE node
I20260809 19:26:01.883096 30180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:01.883388 30180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:01.884543 30180 hybrid_clock.cc:648] HybridClock initialized: now 1786303561884535 us; error 32 us; skew 500 ppm
Aug 09 19:26:01 dist-test-slave-njxd krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303561, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:01.887097 30180 init.cc:377] Logged in from keytab as oryx/127.21.81.195@KRBTEST.COM (short username oryx)
I20260809 19:26:01.887650 30180 webserver.cc:460] Webserver started at http://127.21.81.195:38133/ using document root <none> and password file <none>
I20260809 19:26:01.887893 30180 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:01.887939 30180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:01.888027 30180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:01.889197 30180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "83081c62f0e9449f909154f45694a11b"
format_stamp: "Formatted at 2026-08-09 19:26:01 on dist-test-slave-njxd"
server_key: "b472935a09e2fa9777374e030beab873"
server_key_iv: "457fce9884a5beb2d2f9e535a947f36a"
server_key_version: "encryptionkey@0"
I20260809 19:26:01.889518 30180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "83081c62f0e9449f909154f45694a11b"
format_stamp: "Formatted at 2026-08-09 19:26:01 on dist-test-slave-njxd"
server_key: "b472935a09e2fa9777374e030beab873"
server_key_iv: "457fce9884a5beb2d2f9e535a947f36a"
server_key_version: "encryptionkey@0"
I20260809 19:26:01.891112 30180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:26:01.891779 30195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:01.891930 30180 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:26:01.892031 30180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "83081c62f0e9449f909154f45694a11b"
format_stamp: "Formatted at 2026-08-09 19:26:01 on dist-test-slave-njxd"
server_key: "b472935a09e2fa9777374e030beab873"
server_key_iv: "457fce9884a5beb2d2f9e535a947f36a"
server_key_version: "encryptionkey@0"
I20260809 19:26:01.892110 30180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:01.914801 30180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:01.916708 30180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:01.916874 30180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:01.917415 30180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:01.917477 30180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:01.917520 30180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:01.917546 30180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:26:01 dist-test-slave-njxd krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303561, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
I20260809 19:26:01.924990 30180 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:44731
I20260809 19:26:01.925077 30320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:44731 every 8 connection(s)
I20260809 19:26:01.925369 30180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:26:01.927959 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 30180
I20260809 19:26:01.927992 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.917918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:45239 (local address 127.21.81.254:34111)
0809 19:26:01.917963 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:01.917965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:01.918197 (+ 232us) server_negotiation.cc:407] Connection header received
0809 19:26:01.918727 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:01.918728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:01.918742 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:01.918805 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:01.919679 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.920526 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.921996 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.922353 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.922442 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.924976 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:01.924977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:01.924978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:01.924986 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:01.926584 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.926997 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.926998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.926999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.927063 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.927314 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.927315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.927315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.927437 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:01.927506 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:01.927807 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 19:26:01.927900 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:26:01.928084 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:26:01.928144 30207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.917922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:34111 (local address 127.21.81.195:45239)
0809 19:26:01.918070 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:01.918106 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:01.918627 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:01.918877 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:01.918880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:01.919271 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:01.919561 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.919569 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.920660 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.920661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.921907 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.921910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.922462 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.922463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.922606 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.923156 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:01.923164 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:01.924880 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:01.926689 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:01.926696 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:01.926698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:01.926920 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:01.927132 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:01.927133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:01.927134 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:01.927247 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:01.927511 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:01.927518 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:01.927650 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 19:26:01.927762 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 19:26:01.927921 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260809 19:26:01.928288 21831 external_mini_cluster.cc:1428] Setting key 9e58b97023c8d0bd5d1d642921c09259
I20260809 19:26:01.928836 30321 heartbeater.cc:347] Connected to a master server at 127.21.81.254:34111
I20260809 19:26:01.928918 30321 heartbeater.cc:464] Registering TS with master...
I20260809 19:26:01.929103 30321 heartbeater.cc:511] Master 127.21.81.254:34111 requested a full tablet report, sending...
I20260809 19:26:01.929490 29807 ts_manager.cc:194] Registered new tserver with Master: 83081c62f0e9449f909154f45694a11b (127.21.81.195:44731)
I20260809 19:26:01.930030 29807 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:45239
I20260809 19:26:01.935281 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.929961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:33981 (local address 127.21.81.254:34111)
0809 19:26:01.930158 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:01.930161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:01.930199 (+ 38us) server_negotiation.cc:407] Connection header received
0809 19:26:01.930265 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:01.930266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:01.930277 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:01.930332 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:01.930653 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.931097 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.931784 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.931981 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.932051 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.932998 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:01.932999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:01.933000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:01.933007 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:01.934200 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.934517 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.934518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.934519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.934583 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.934780 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.934781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.934781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.934889 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:01.934961 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:01.935119 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 19:26:01.935194 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176}
I20260809 19:26:01.935299 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.929879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:34111 (local address 127.21.81.195:33981)
0809 19:26:01.930130 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:01.930143 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:01.930213 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:01.930343 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:01.930343 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:01.930510 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:01.930566 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.930568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.931185 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.931186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.931704 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.931705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.932050 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.932051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.932116 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.932551 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:01.932557 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:01.932907 (+ 350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:01.934278 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:01.934279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:01.934279 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:01.934448 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:01.934646 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:01.934647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:01.934648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:01.934717 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:01.934959 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:01.934960 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:01.935006 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 19:26:01.935081 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 19:26:01.935188 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":99,"threads_started":1}
I20260809 19:26:01.939906 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 19:26:01 dist-test-slave-njxd krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303561, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 19:26:01 dist-test-slave-njxd krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:01 dist-test-slave-njxd krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 19:26:01.962077 29858 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:01.956513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47918 (local address 127.21.81.254:34111)
0809 19:26:01.956593 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:01.956596 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:01.956630 (+ 34us) server_negotiation.cc:407] Connection header received
0809 19:26:01.956733 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:01.956735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:01.956747 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:01.956808 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:01.957368 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.957941 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.958657 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.958838 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.958890 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.961977 (+ 3087us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47918: BlockingRecv error: recv got EOF from 127.0.0.1:47918 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 19:26:01 dist-test-slave-njxd krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303561, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
I20260809 19:26:01.969919 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.962884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47924 (local address 127.21.81.254:34111)
0809 19:26:01.962954 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:01.962956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:01.963048 (+ 92us) server_negotiation.cc:407] Connection header received
0809 19:26:01.963126 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:01.963127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:01.963142 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:01.963204 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:01.963771 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.964235 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.964956 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.965140 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.965211 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.967261 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:01.967263 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:01.967263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:01.967270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:01.968726 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.969113 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.969115 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.969115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.969174 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:01.969363 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:01.969364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:01.969364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:01.969478 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:01.969581 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:01.969737 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 19:26:01.969815 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:26:01.971539 29807 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:26:01.973142 29807 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:26:01.979867 30345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.975627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:38535 (local address 127.0.0.1:41552)
0809 19:26:01.976059 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:01.976072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:01.976136 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:01.976413 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:01.976414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:01.976417 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:26:01.976473 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.976475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.977351 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.977352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.978647 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.978650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.979552 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.979553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.979650 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.979665 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 19:26:01.979739 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 19:26:01.979793 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":374,"threads_started":1}
I20260809 19:26:01.979918 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.975787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41552 (local address 127.21.81.194:38535)
0809 19:26:01.975963 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:01.975968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:01.976147 (+ 179us) server_negotiation.cc:407] Connection header received
0809 19:26:01.976242 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:01.976244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:01.976261 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:01.976335 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:26:01.976545 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.977264 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.978737 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.979461 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.979543 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.979670 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:01.979759 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 19:26:01.979800 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260809 19:26:01.980042 30349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.975611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:42947 (local address 127.0.0.1:44026)
0809 19:26:01.976038 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:01.976046 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:01.976090 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:01.976339 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:01.976342 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:01.976344 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:26:01.976430 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.976433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.977795 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.977796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.978816 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.978818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.979794 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.979795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.979862 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.979867 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:26:01.979918 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:26:01.979962 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":12,"thread_start_us":118,"threads_started":1}
I20260809 19:26:01.980262 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.975902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44026 (local address 127.21.81.193:42947)
0809 19:26:01.976092 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:01.976096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:01.976108 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:26:01.976152 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:01.976154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:01.976171 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:01.976249 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:26:01.976501 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.977713 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.978886 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.979614 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.979691 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.979758 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:01.980113 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 19:26:01.980148 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":79,"threads_started":1}
I20260809 19:26:01.980852 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.975611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:44731 (local address 127.0.0.1:42426)
0809 19:26:01.975808 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:01.975848 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:01.975917 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:01.976515 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:01.976516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:01.976518 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:26:01.976561 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.976562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.978885 (+ 2323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.978885 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.979549 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:01.979551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.980607 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.980608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:01.980683 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.980687 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:26:01.980744 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 19:26:01.980782 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":77,"threads_started":1}
I20260809 19:26:01.981011 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:01.975703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42426 (local address 127.21.81.195:44731)
0809 19:26:01.976302 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:01.976306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:01.976316 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:26:01.976353 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:01.976355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:01.976369 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:01.976444 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:26:01.976647 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.977714 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.979664 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:01.980417 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:01.980499 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:01.980572 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:01.980774 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 19:26:01.980819 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":93,"threads_started":1}
I20260809 19:26:01.980929 30099 tablet_service.cc:1467] Processing CreateTablet for tablet 5aadea42ecc0479aada72bf9472394eb (DEFAULT_TABLE table=test-table [id=024a2385770b4ada9c2b26a92f97739a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:26:01.980926 29952 tablet_service.cc:1467] Processing CreateTablet for tablet 5aadea42ecc0479aada72bf9472394eb (DEFAULT_TABLE table=test-table [id=024a2385770b4ada9c2b26a92f97739a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:26:01.981259 30099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5aadea42ecc0479aada72bf9472394eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:01.981273 29952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5aadea42ecc0479aada72bf9472394eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:01.982196 30250 tablet_service.cc:1467] Processing CreateTablet for tablet 5aadea42ecc0479aada72bf9472394eb (DEFAULT_TABLE table=test-table [id=024a2385770b4ada9c2b26a92f97739a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:26:01.982513 30250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5aadea42ecc0479aada72bf9472394eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:01.984448 30350 tablet_bootstrap.cc:492] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca: Bootstrap starting.
I20260809 19:26:01.985157 30350 tablet_bootstrap.cc:654] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:01.985424 30351 tablet_bootstrap.cc:492] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46: Bootstrap starting.
I20260809 19:26:01.985462 30350 log.cc:824] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca: Log is configured to *not* fsync() on all Append() calls
I20260809 19:26:01.986129 30351 tablet_bootstrap.cc:654] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:01.986213 30350 tablet_bootstrap.cc:492] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca: No bootstrap required, opened a new log
I20260809 19:26:01.986287 30350 ts_tablet_manager.cc:1397] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:26:01.986496 30351 log.cc:824] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46: Log is configured to *not* fsync() on all Append() calls
I20260809 19:26:01.986891 30353 tablet_bootstrap.cc:492] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b: Bootstrap starting.
I20260809 19:26:01.987291 30351 tablet_bootstrap.cc:492] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46: No bootstrap required, opened a new log
I20260809 19:26:01.987385 30351 ts_tablet_manager.cc:1397] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:26:01.987522 30353 tablet_bootstrap.cc:654] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:01.987845 30353 log.cc:824] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:26:01.988313 30350 raft_consensus.cc:348] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } }
I20260809 19:26:01.988471 30350 raft_consensus.cc:374] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:01.988511 30350 raft_consensus.cc:729] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24014678d6a345abaf05bbdb3d0003ca, State: Initialized, Role: FOLLOWER
I20260809 19:26:01.988617 30353 tablet_bootstrap.cc:492] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b: No bootstrap required, opened a new log
I20260809 19:26:01.988682 30353 ts_tablet_manager.cc:1397] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:26:01.988621 30350 consensus_queue.cc:260] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } }
I20260809 19:26:01.988955 30170 heartbeater.cc:503] Master 127.21.81.254:34111 was elected leader, sending a full tablet report...
I20260809 19:26:01.988936 30350 ts_tablet_manager.cc:1428] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:26:01.989609 30351 raft_consensus.cc:348] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } }
I20260809 19:26:01.989771 30351 raft_consensus.cc:374] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:01.989812 30351 raft_consensus.cc:729] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a6da08dd544f4c8a4206925dd4fa46, State: Initialized, Role: FOLLOWER
I20260809 19:26:01.989956 30351 consensus_queue.cc:260] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } }
I20260809 19:26:01.990281 30351 ts_tablet_manager.cc:1428] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:26:01.990401 30022 heartbeater.cc:503] Master 127.21.81.254:34111 was elected leader, sending a full tablet report...
I20260809 19:26:01.990875 30353 raft_consensus.cc:348] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } }
I20260809 19:26:01.991056 30353 raft_consensus.cc:374] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:01.991118 30353 raft_consensus.cc:729] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83081c62f0e9449f909154f45694a11b, State: Initialized, Role: FOLLOWER
I20260809 19:26:01.991233 30353 consensus_queue.cc:260] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } }
I20260809 19:26:01.991484 30353 ts_tablet_manager.cc:1428] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:26:01.991546 30321 heartbeater.cc:503] Master 127.21.81.254:34111 was elected leader, sending a full tablet report...
W20260809 19:26:02.011005 30172 tablet.cc:2369] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:26:02.087160 30023 tablet.cc:2369] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:26:02.111809 30357 raft_consensus.cc:482] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:26:02.111935 30357 raft_consensus.cc:504] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } }
I20260809 19:26:02.112298 30357 leader_election.cc:288] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83081c62f0e9449f909154f45694a11b (127.21.81.195:44731), 24014678d6a345abaf05bbdb3d0003ca (127.21.81.194:38535)
I20260809 19:26:02.115264 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.112592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:33101 (local address 127.21.81.194:38535)
0809 19:26:02.112642 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:02.112644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:02.112734 (+ 90us) server_negotiation.cc:407] Connection header received
0809 19:26:02.112807 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:02.112808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:02.112816 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:02.112882 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:26:02.113066 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.113518 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.114552 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.114895 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.114950 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.114979 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:02.115136 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 19:26:02.115185 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:26:02.115279 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.112556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:35035 (local address 127.21.81.195:44731)
0809 19:26:02.112606 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:02.112608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:02.112670 (+ 62us) server_negotiation.cc:407] Connection header received
0809 19:26:02.112788 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:02.112789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:02.112800 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:02.112891 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:26:02.113033 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.113518 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.114515 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.114902 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.114988 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.114996 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:02.115161 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 19:26:02.115198 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:26:02.115340 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.112508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:38535 (local address 127.21.81.193:33101)
0809 19:26:02.112658 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:02.112668 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:02.112744 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:02.112891 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:02.112892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:02.112894 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:26:02.112983 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.112985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.113621 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.113622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.114435 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.114438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.114979 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.114980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.115066 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.115070 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:26:02.115119 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:26:02.115180 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":52,"thread_start_us":77,"threads_started":1}
I20260809 19:26:02.115521 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.112452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:44731 (local address 127.21.81.193:35035)
0809 19:26:02.112592 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:02.112604 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:02.112694 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:02.112898 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:02.112899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:02.112901 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:26:02.112961 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.112963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.113625 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.113626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.114429 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.114431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.115010 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.115011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.115078 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.115081 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:26:02.115146 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 19:26:02.115186 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":1,"thread_start_us":67,"threads_started":1}
I20260809 19:26:02.115700 30122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aadea42ecc0479aada72bf9472394eb" candidate_uuid: "30a6da08dd544f4c8a4206925dd4fa46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24014678d6a345abaf05bbdb3d0003ca" is_pre_election: true
I20260809 19:26:02.115895 30122 raft_consensus.cc:2454] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30a6da08dd544f4c8a4206925dd4fa46 in term 0.
I20260809 19:26:02.116008 30271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aadea42ecc0479aada72bf9472394eb" candidate_uuid: "30a6da08dd544f4c8a4206925dd4fa46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83081c62f0e9449f909154f45694a11b" is_pre_election: true
I20260809 19:26:02.116142 29900 leader_election.cc:302] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24014678d6a345abaf05bbdb3d0003ca, 30a6da08dd544f4c8a4206925dd4fa46; no voters:
I20260809 19:26:02.116161 30271 raft_consensus.cc:2454] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30a6da08dd544f4c8a4206925dd4fa46 in term 0.
I20260809 19:26:02.116369 30357 raft_consensus.cc:2781] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:26:02.116425 30357 raft_consensus.cc:482] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:26:02.116441 30357 raft_consensus.cc:3037] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:02.117138 30357 raft_consensus.cc:504] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } }
I20260809 19:26:02.117354 30357 leader_election.cc:288] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [CANDIDATE]: Term 1 election: Requested vote from peers 83081c62f0e9449f909154f45694a11b (127.21.81.195:44731), 24014678d6a345abaf05bbdb3d0003ca (127.21.81.194:38535)
I20260809 19:26:02.117425 30271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aadea42ecc0479aada72bf9472394eb" candidate_uuid: "30a6da08dd544f4c8a4206925dd4fa46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83081c62f0e9449f909154f45694a11b"
I20260809 19:26:02.117516 30122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aadea42ecc0479aada72bf9472394eb" candidate_uuid: "30a6da08dd544f4c8a4206925dd4fa46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24014678d6a345abaf05bbdb3d0003ca"
I20260809 19:26:02.117527 30271 raft_consensus.cc:3037] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:02.117635 30122 raft_consensus.cc:3037] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:02.118316 30271 raft_consensus.cc:2454] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30a6da08dd544f4c8a4206925dd4fa46 in term 1.
I20260809 19:26:02.118371 30122 raft_consensus.cc:2454] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30a6da08dd544f4c8a4206925dd4fa46 in term 1.
I20260809 19:26:02.118541 29903 leader_election.cc:302] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30a6da08dd544f4c8a4206925dd4fa46, 83081c62f0e9449f909154f45694a11b; no voters:
I20260809 19:26:02.118676 30357 raft_consensus.cc:2781] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:26:02.118819 30357 raft_consensus.cc:686] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 LEADER]: Becoming Leader. State: Replica: 30a6da08dd544f4c8a4206925dd4fa46, State: Running, Role: LEADER
I20260809 19:26:02.118938 30357 consensus_queue.cc:237] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } }
I20260809 19:26:02.119758 29806 catalog_manager.cc:5495] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30a6da08dd544f4c8a4206925dd4fa46 (127.21.81.193). New cstate: current_term: 1 leader_uuid: "30a6da08dd544f4c8a4206925dd4fa46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260809 19:26:02.126434 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.123986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44030 (local address 127.21.81.193:42947)
0809 19:26:02.124040 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:02.124042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:02.124050 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:26:02.124081 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:02.124082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:02.124098 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:02.124179 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:26:02.124319 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.124847 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.125675 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.125871 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.125952 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.126123 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:26:02.126225 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:26:02.126311 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:02.126334 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 19:26:02.126354 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 19:26:02.128090 30122 raft_consensus.cc:1261] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca [term 1 FOLLOWER]: Refusing update from remote peer 30a6da08dd544f4c8a4206925dd4fa46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:26:02.128098 30271 raft_consensus.cc:1261] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Refusing update from remote peer 30a6da08dd544f4c8a4206925dd4fa46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:26:02.128414 30357 consensus_queue.cc:1035] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:26:02.128538 30361 consensus_queue.cc:1035] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:26:02.130826 30364 mvcc.cc:205] Tried to move back new op lower bound from 7316699390474272768 to 7316699390439845888. Current Snapshot: MvccSnapshot[applied={T|T < 7316699390474272768}]
I20260809 19:26:02.132973 29807 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:26:02.133093 29807 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:26:02.133817 29807 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a: Sending DeleteTablet for 3 replicas of tablet 5aadea42ecc0479aada72bf9472394eb
I20260809 19:26:02.134122 30250 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aadea42ecc0479aada72bf9472394eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:02 UTC) from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:42426
I20260809 19:26:02.134131 29952 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aadea42ecc0479aada72bf9472394eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:02 UTC) from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:44026
I20260809 19:26:02.134136 30099 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aadea42ecc0479aada72bf9472394eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:02 UTC) from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:41552
I20260809 19:26:02.134454 30374 tablet_replica.cc:330] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46: stopping tablet replica
I20260809 19:26:02.134579 30374 raft_consensus.cc:2229] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:26:02.134722 30374 raft_consensus.cc:2258] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:02.134694 29807 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:26:02.134837 29807 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:26:02.134893 30376 tablet_replica.cc:330] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca: stopping tablet replica
I20260809 19:26:02.134994 30376 raft_consensus.cc:2229] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:26:02.135396 30376 raft_consensus.cc:2258] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:02.135941 30374 ts_tablet_manager.cc:1905] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:26:02.135993 30376 ts_tablet_manager.cc:1905] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:26:02.136330 30099 tablet_service.cc:1467] Processing CreateTablet for tablet 698bd93ed6e149e29c0744409e828651 (DEFAULT_TABLE table=test-table [id=1c17d1b21bdb4cfab0b5a7105ed2d433]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:26:02.136459 30099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 698bd93ed6e149e29c0744409e828651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:02.137467 29952 tablet_service.cc:1467] Processing CreateTablet for tablet 698bd93ed6e149e29c0744409e828651 (DEFAULT_TABLE table=test-table [id=1c17d1b21bdb4cfab0b5a7105ed2d433]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:26:02.137601 29952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 698bd93ed6e149e29c0744409e828651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:02.138010 30376 ts_tablet_manager.cc:1918] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:26:02.138084 30376 log.cc:1196] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/5aadea42ecc0479aada72bf9472394eb
I20260809 19:26:02.138237 30374 ts_tablet_manager.cc:1918] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:26:02.138365 30374 log.cc:1196] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/wal/wals/5aadea42ecc0479aada72bf9472394eb
I20260809 19:26:02.138379 30376 ts_tablet_manager.cc:1939] T 5aadea42ecc0479aada72bf9472394eb P 24014678d6a345abaf05bbdb3d0003ca: Deleting consensus metadata
I20260809 19:26:02.138713 30374 ts_tablet_manager.cc:1939] T 5aadea42ecc0479aada72bf9472394eb P 30a6da08dd544f4c8a4206925dd4fa46: Deleting consensus metadata
I20260809 19:26:02.138760 30375 tablet_replica.cc:330] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b: stopping tablet replica
I20260809 19:26:02.138877 30375 raft_consensus.cc:2229] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:26:02.138923 29791 catalog_manager.cc:4841] TS 24014678d6a345abaf05bbdb3d0003ca (127.21.81.194:38535): tablet 5aadea42ecc0479aada72bf9472394eb (table test-table [id=024a2385770b4ada9c2b26a92f97739a]) successfully deleted
I20260809 19:26:02.138995 30351 tablet_bootstrap.cc:492] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46: Bootstrap starting.
I20260809 19:26:02.138996 30375 raft_consensus.cc:2258] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:02.139119 30250 tablet_service.cc:1467] Processing CreateTablet for tablet 698bd93ed6e149e29c0744409e828651 (DEFAULT_TABLE table=test-table [id=1c17d1b21bdb4cfab0b5a7105ed2d433]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:26:02.139220 30250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 698bd93ed6e149e29c0744409e828651. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:02.139427 30351 tablet_bootstrap.cc:654] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:02.139467 30375 ts_tablet_manager.cc:1905] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:26:02.140617 30353 tablet_bootstrap.cc:492] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b: Bootstrap starting.
I20260809 19:26:02.140753 29792 catalog_manager.cc:4841] TS 30a6da08dd544f4c8a4206925dd4fa46 (127.21.81.193:42947): tablet 5aadea42ecc0479aada72bf9472394eb (table test-table [id=024a2385770b4ada9c2b26a92f97739a]) successfully deleted
I20260809 19:26:02.141049 30353 tablet_bootstrap.cc:654] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:02.141332 30375 ts_tablet_manager.cc:1918] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:26:02.141393 30375 log.cc:1196] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/5aadea42ecc0479aada72bf9472394eb
I20260809 19:26:02.141701 30375 ts_tablet_manager.cc:1939] T 5aadea42ecc0479aada72bf9472394eb P 83081c62f0e9449f909154f45694a11b: Deleting consensus metadata
I20260809 19:26:02.141813 30353 tablet_bootstrap.cc:492] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b: No bootstrap required, opened a new log
I20260809 19:26:02.141858 30353 ts_tablet_manager.cc:1397] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:26:02.141870 30350 tablet_bootstrap.cc:492] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca: Bootstrap starting.
I20260809 19:26:02.142002 30353 raft_consensus.cc:348] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.142066 30353 raft_consensus.cc:374] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:02.142087 30353 raft_consensus.cc:729] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83081c62f0e9449f909154f45694a11b, State: Initialized, Role: FOLLOWER
I20260809 19:26:02.142141 30353 consensus_queue.cc:260] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.142233 30353 ts_tablet_manager.cc:1428] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:02.142278 30350 tablet_bootstrap.cc:654] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:02.142869 29793 catalog_manager.cc:4841] TS 83081c62f0e9449f909154f45694a11b (127.21.81.195:44731): tablet 5aadea42ecc0479aada72bf9472394eb (table test-table [id=024a2385770b4ada9c2b26a92f97739a]) successfully deleted
I20260809 19:26:02.143227 30350 tablet_bootstrap.cc:492] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca: No bootstrap required, opened a new log
I20260809 19:26:02.143268 30350 ts_tablet_manager.cc:1397] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:26:02.143435 30350 raft_consensus.cc:348] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.143487 30350 raft_consensus.cc:374] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:02.143500 30350 raft_consensus.cc:729] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24014678d6a345abaf05bbdb3d0003ca, State: Initialized, Role: FOLLOWER
I20260809 19:26:02.143546 30350 consensus_queue.cc:260] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.143632 30350 ts_tablet_manager.cc:1428] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:26:02.144104 30351 tablet_bootstrap.cc:492] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46: No bootstrap required, opened a new log
I20260809 19:26:02.144157 30351 ts_tablet_manager.cc:1397] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 19:26:02.144294 30351 raft_consensus.cc:348] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.144375 30351 raft_consensus.cc:374] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:02.144395 30351 raft_consensus.cc:729] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a6da08dd544f4c8a4206925dd4fa46, State: Initialized, Role: FOLLOWER
I20260809 19:26:02.144455 30351 consensus_queue.cc:260] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.144546 30351 ts_tablet_manager.cc:1428] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260809 19:26:02.176030 30322 tablet.cc:2369] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:26:02.295998 30356 raft_consensus.cc:482] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:26:02.296123 30356 raft_consensus.cc:504] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.296494 30356 leader_election.cc:288] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30a6da08dd544f4c8a4206925dd4fa46 (127.21.81.193:42947), 83081c62f0e9449f909154f45694a11b (127.21.81.195:44731)
I20260809 19:26:02.299562 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.296824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:54099 (local address 127.21.81.193:42947)
0809 19:26:02.296875 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:02.296878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:02.296987 (+ 109us) server_negotiation.cc:407] Connection header received
0809 19:26:02.297072 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:02.297098 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:02.297108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:02.297177 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:26:02.297346 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.297852 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.298840 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.299207 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.299288 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.299297 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:02.299446 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 19:26:02.299488 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:26:02.299571 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.296798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:58795 (local address 127.21.81.195:44731)
0809 19:26:02.296864 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:02.296866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:02.296926 (+ 60us) server_negotiation.cc:407] Connection header received
0809 19:26:02.297017 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:02.297018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:02.297032 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:02.297097 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:26:02.297285 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.297830 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.298829 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.299220 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.299302 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.299310 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:02.299456 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 19:26:02.299494 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:26:02.299600 30381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.296708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:44731 (local address 127.21.81.194:58795)
0809 19:26:02.296863 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:02.296875 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:02.296945 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:02.297104 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:02.297105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:02.297108 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:26:02.297192 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.297194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.297921 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.297922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.298733 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.298735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.299303 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.299304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.299377 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.299386 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 19:26:02.299450 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 19:26:02.299493 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":79,"threads_started":1}
I20260809 19:26:02.299616 30380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.296708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:42947 (local address 127.21.81.194:54099)
0809 19:26:02.296901 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:02.296911 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:02.296999 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:02.297176 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:02.297177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:02.297180 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:26:02.297263 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.297265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.297933 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.297934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.298746 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.298748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.299287 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.299289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.299364 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.299367 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:26:02.299427 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:26:02.299473 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":62,"thread_start_us":56,"threads_started":1}
I20260809 19:26:02.299853 30271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "698bd93ed6e149e29c0744409e828651" candidate_uuid: "24014678d6a345abaf05bbdb3d0003ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83081c62f0e9449f909154f45694a11b" is_pre_election: true
I20260809 19:26:02.299959 30271 raft_consensus.cc:2454] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24014678d6a345abaf05bbdb3d0003ca in term 0.
I20260809 19:26:02.299942 29972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "698bd93ed6e149e29c0744409e828651" candidate_uuid: "24014678d6a345abaf05bbdb3d0003ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30a6da08dd544f4c8a4206925dd4fa46" is_pre_election: true
I20260809 19:26:02.300094 29972 raft_consensus.cc:2454] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24014678d6a345abaf05bbdb3d0003ca in term 0.
I20260809 19:26:02.300196 30051 leader_election.cc:302] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24014678d6a345abaf05bbdb3d0003ca, 83081c62f0e9449f909154f45694a11b; no voters:
I20260809 19:26:02.300349 30356 raft_consensus.cc:2781] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:26:02.300415 30356 raft_consensus.cc:482] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:26:02.300449 30356 raft_consensus.cc:3037] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:02.301177 30356 raft_consensus.cc:504] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.301332 30356 leader_election.cc:288] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [CANDIDATE]: Term 1 election: Requested vote from peers 30a6da08dd544f4c8a4206925dd4fa46 (127.21.81.193:42947), 83081c62f0e9449f909154f45694a11b (127.21.81.195:44731)
I20260809 19:26:02.301494 29972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "698bd93ed6e149e29c0744409e828651" candidate_uuid: "24014678d6a345abaf05bbdb3d0003ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30a6da08dd544f4c8a4206925dd4fa46"
I20260809 19:26:02.301497 30271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "698bd93ed6e149e29c0744409e828651" candidate_uuid: "24014678d6a345abaf05bbdb3d0003ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83081c62f0e9449f909154f45694a11b"
I20260809 19:26:02.301614 30271 raft_consensus.cc:3037] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:02.301623 29972 raft_consensus.cc:3037] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:02.302240 30271 raft_consensus.cc:2454] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24014678d6a345abaf05bbdb3d0003ca in term 1.
I20260809 19:26:02.302346 29972 raft_consensus.cc:2454] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24014678d6a345abaf05bbdb3d0003ca in term 1.
I20260809 19:26:02.302407 30051 leader_election.cc:302] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24014678d6a345abaf05bbdb3d0003ca, 83081c62f0e9449f909154f45694a11b; no voters:
I20260809 19:26:02.302500 30356 raft_consensus.cc:2781] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:26:02.302611 30356 raft_consensus.cc:686] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [term 1 LEADER]: Becoming Leader. State: Replica: 24014678d6a345abaf05bbdb3d0003ca, State: Running, Role: LEADER
I20260809 19:26:02.302698 30356 consensus_queue.cc:237] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.303428 29807 catalog_manager.cc:5495] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 24014678d6a345abaf05bbdb3d0003ca (127.21.81.194). New cstate: current_term: 1 leader_uuid: "24014678d6a345abaf05bbdb3d0003ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } health_report { overall_health: UNKNOWN } } }
Aug 09 19:26:02 dist-test-slave-njxd krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303561, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.254@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
I20260809 19:26:02.365386 30391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.358338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:34111 (local address 127.0.0.1:47936)
0809 19:26:02.358535 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:02.358545 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:02.358635 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:02.358796 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:02.358797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:02.359132 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:02.359184 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.359187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.359865 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.359866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.360389 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.360391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.360741 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.360742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.360801 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.361316 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:02.361321 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:02.362858 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:02.364385 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:02.364391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:02.364394 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:02.364617 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:02.364813 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:02.364814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:02.364814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:02.364870 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:02.365078 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:02.365084 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:02.365139 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 19:26:02.365187 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 19:26:02.365282 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
I20260809 19:26:02.365425 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.358404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47936 (local address 127.21.81.254:34111)
0809 19:26:02.358488 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:02.358491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:02.358616 (+ 125us) server_negotiation.cc:407] Connection header received
0809 19:26:02.358714 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:02.358715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:02.358725 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:02.358792 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:02.359270 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.359785 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.360473 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.360656 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.360739 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.362936 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:02.362937 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:02.362937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:02.362944 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:02.364311 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:02.364687 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:02.364688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:02.364688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:02.364750 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:02.364915 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:02.364916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:02.364916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:02.365020 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:02.365067 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:02.365212 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 19:26:02.365329 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":48}
I20260809 19:26:02.367523 29807 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:47936:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 19:26:02.367740 29807 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:26:02.369207 30099 tablet_service.cc:1467] Processing CreateTablet for tablet 5d758d71f9ac46bbbbfe544dcaf3bf50 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6c6c12c82814191b68beb3b3112e693]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:26:02.369231 30250 tablet_service.cc:1467] Processing CreateTablet for tablet 5d758d71f9ac46bbbbfe544dcaf3bf50 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6c6c12c82814191b68beb3b3112e693]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:26:02.369230 29952 tablet_service.cc:1467] Processing CreateTablet for tablet 5d758d71f9ac46bbbbfe544dcaf3bf50 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6c6c12c82814191b68beb3b3112e693]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:26:02.369364 29952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d758d71f9ac46bbbbfe544dcaf3bf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:02.369369 30099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d758d71f9ac46bbbbfe544dcaf3bf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:02.369369 30250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d758d71f9ac46bbbbfe544dcaf3bf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:02.370918 30351 tablet_bootstrap.cc:492] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46: Bootstrap starting.
I20260809 19:26:02.370942 30350 tablet_bootstrap.cc:492] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca: Bootstrap starting.
I20260809 19:26:02.370942 30353 tablet_bootstrap.cc:492] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b: Bootstrap starting.
I20260809 19:26:02.371223 30351 tablet_bootstrap.cc:654] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:02.371394 30350 tablet_bootstrap.cc:654] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:02.371394 30353 tablet_bootstrap.cc:654] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:02.372108 30350 tablet_bootstrap.cc:492] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca: No bootstrap required, opened a new log
I20260809 19:26:02.372185 30350 ts_tablet_manager.cc:1397] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:26:02.372257 30351 tablet_bootstrap.cc:492] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46: No bootstrap required, opened a new log
I20260809 19:26:02.372314 30351 ts_tablet_manager.cc:1397] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:26:02.372355 30350 raft_consensus.cc:348] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.372430 30350 raft_consensus.cc:374] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:02.372462 30350 raft_consensus.cc:729] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24014678d6a345abaf05bbdb3d0003ca, State: Initialized, Role: FOLLOWER
I20260809 19:26:02.372437 30351 raft_consensus.cc:348] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.372524 30351 raft_consensus.cc:374] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:02.372514 30350 consensus_queue.cc:260] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.372560 30351 raft_consensus.cc:729] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a6da08dd544f4c8a4206925dd4fa46, State: Initialized, Role: FOLLOWER
I20260809 19:26:02.372608 30350 ts_tablet_manager.cc:1428] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:26:02.372610 30351 consensus_queue.cc:260] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.372715 30351 ts_tablet_manager.cc:1428] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:26:02.372735 30356 tablet_replica.cc:439] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } }
I20260809 19:26:02.372745 30361 tablet_replica.cc:439] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } }
I20260809 19:26:02.372802 30356 tablet_replica.cc:442] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:26:02.372831 30361 tablet_replica.cc:442] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:26:02.373212 30353 tablet_bootstrap.cc:492] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b: No bootstrap required, opened a new log
I20260809 19:26:02.373287 30353 ts_tablet_manager.cc:1397] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:26:02.373458 30353 raft_consensus.cc:348] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.373513 30353 raft_consensus.cc:374] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:02.373526 30353 raft_consensus.cc:729] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83081c62f0e9449f909154f45694a11b, State: Initialized, Role: FOLLOWER
I20260809 19:26:02.373606 30353 consensus_queue.cc:260] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.373744 30353 ts_tablet_manager.cc:1428] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:02.373807 30358 tablet_replica.cc:439] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } }
I20260809 19:26:02.373914 30358 tablet_replica.cc:442] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:26:02.422079 30361 raft_consensus.cc:482] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:26:02.422240 30361 raft_consensus.cc:504] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.422431 30361 leader_election.cc:288] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24014678d6a345abaf05bbdb3d0003ca (127.21.81.194:38535), 83081c62f0e9449f909154f45694a11b (127.21.81.195:44731)
I20260809 19:26:02.422669 30122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d758d71f9ac46bbbbfe544dcaf3bf50" candidate_uuid: "30a6da08dd544f4c8a4206925dd4fa46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24014678d6a345abaf05bbdb3d0003ca" is_pre_election: true
I20260809 19:26:02.422704 30271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d758d71f9ac46bbbbfe544dcaf3bf50" candidate_uuid: "30a6da08dd544f4c8a4206925dd4fa46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83081c62f0e9449f909154f45694a11b" is_pre_election: true
I20260809 19:26:02.422781 30122 raft_consensus.cc:2454] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30a6da08dd544f4c8a4206925dd4fa46 in term 0.
I20260809 19:26:02.422811 30271 raft_consensus.cc:2454] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30a6da08dd544f4c8a4206925dd4fa46 in term 0.
I20260809 19:26:02.422995 29900 leader_election.cc:302] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24014678d6a345abaf05bbdb3d0003ca, 30a6da08dd544f4c8a4206925dd4fa46; no voters:
I20260809 19:26:02.423128 30361 raft_consensus.cc:2781] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:26:02.423172 30361 raft_consensus.cc:482] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:26:02.423192 30361 raft_consensus.cc:3037] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:02.423916 30361 raft_consensus.cc:504] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.424103 30361 leader_election.cc:288] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [CANDIDATE]: Term 1 election: Requested vote from peers 24014678d6a345abaf05bbdb3d0003ca (127.21.81.194:38535), 83081c62f0e9449f909154f45694a11b (127.21.81.195:44731)
I20260809 19:26:02.424279 30271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d758d71f9ac46bbbbfe544dcaf3bf50" candidate_uuid: "30a6da08dd544f4c8a4206925dd4fa46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83081c62f0e9449f909154f45694a11b"
I20260809 19:26:02.424290 30122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d758d71f9ac46bbbbfe544dcaf3bf50" candidate_uuid: "30a6da08dd544f4c8a4206925dd4fa46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24014678d6a345abaf05bbdb3d0003ca"
I20260809 19:26:02.424369 30122 raft_consensus.cc:3037] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:02.424378 30271 raft_consensus.cc:3037] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:02.425051 30122 raft_consensus.cc:2454] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30a6da08dd544f4c8a4206925dd4fa46 in term 1.
I20260809 19:26:02.425060 30271 raft_consensus.cc:2454] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30a6da08dd544f4c8a4206925dd4fa46 in term 1.
I20260809 19:26:02.425225 29900 leader_election.cc:302] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24014678d6a345abaf05bbdb3d0003ca, 30a6da08dd544f4c8a4206925dd4fa46; no voters:
I20260809 19:26:02.425315 30361 raft_consensus.cc:2781] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:26:02.425390 30361 raft_consensus.cc:686] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 LEADER]: Becoming Leader. State: Replica: 30a6da08dd544f4c8a4206925dd4fa46, State: Running, Role: LEADER
I20260809 19:26:02.425444 30361 consensus_queue.cc:237] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.425800 30357 tablet_replica.cc:439] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46: TxnStatusTablet state changed. Reason: New leader 30a6da08dd544f4c8a4206925dd4fa46. Latest consensus state: current_term: 1 leader_uuid: "30a6da08dd544f4c8a4206925dd4fa46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } }
I20260809 19:26:02.425894 30357 tablet_replica.cc:442] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:26:02.426141 29807 catalog_manager.cc:5495] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30a6da08dd544f4c8a4206925dd4fa46 (127.21.81.193). New cstate: current_term: 1 leader_uuid: "30a6da08dd544f4c8a4206925dd4fa46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } health_report { overall_health: UNKNOWN } } }
I20260809 19:26:02.426481 30396 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 19:26:02.426625 30396 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 19:26:02.431584 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.429163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44044 (local address 127.21.81.193:42947)
0809 19:26:02.429249 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:02.429252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:02.429284 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:26:02.429368 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:02.429369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:02.429380 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:02.429443 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:26:02.429611 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.430071 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.430846 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.431055 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.431138 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.431318 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:26:02.431363 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:26:02.431421 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:02.431494 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 19:26:02.431518 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 19:26:02.431630 30391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.429124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:42947 (local address 127.0.0.1:44044)
0809 19:26:02.429238 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:02.429250 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:02.429306 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:02.429458 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:02.429459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:02.429460 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:26:02.429512 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.429515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.430176 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.430177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.430759 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.430761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.431138 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.431139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.431208 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.431229 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:26:02.431425 (+ 196us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:26:02.431436 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 19:26:02.431489 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 19:26:02.431542 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 19:26:02.432652 30122 raft_consensus.cc:1261] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca [term 1 FOLLOWER]: Refusing update from remote peer 30a6da08dd544f4c8a4206925dd4fa46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:26:02.432651 30271 raft_consensus.cc:1261] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Refusing update from remote peer 30a6da08dd544f4c8a4206925dd4fa46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:26:02.432888 30361 consensus_queue.cc:1035] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:26:02.432963 30357 consensus_queue.cc:1035] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:26:02.433491 30358 tablet_replica.cc:439] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b: TxnStatusTablet state changed. Reason: New leader 30a6da08dd544f4c8a4206925dd4fa46. Latest consensus state: current_term: 1 leader_uuid: "30a6da08dd544f4c8a4206925dd4fa46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } }
I20260809 19:26:02.433604 30358 tablet_replica.cc:442] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:26:02.433547 30356 tablet_replica.cc:439] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca: TxnStatusTablet state changed. Reason: New leader 30a6da08dd544f4c8a4206925dd4fa46. Latest consensus state: current_term: 1 leader_uuid: "30a6da08dd544f4c8a4206925dd4fa46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } }
I20260809 19:26:02.433952 30356 tablet_replica.cc:442] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:26:02.434273 30361 tablet_replica.cc:439] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30a6da08dd544f4c8a4206925dd4fa46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } }
I20260809 19:26:02.434351 30361 tablet_replica.cc:442] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:26:02.434628 30358 tablet_replica.cc:439] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30a6da08dd544f4c8a4206925dd4fa46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } }
I20260809 19:26:02.434712 30358 tablet_replica.cc:442] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 83081c62f0e9449f909154f45694a11b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:26:02.436120 30357 tablet_replica.cc:439] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30a6da08dd544f4c8a4206925dd4fa46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } }
I20260809 19:26:02.436218 30357 tablet_replica.cc:442] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 30a6da08dd544f4c8a4206925dd4fa46: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:26:02.436620 30356 tablet_replica.cc:439] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30a6da08dd544f4c8a4206925dd4fa46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } } }
I20260809 19:26:02.436704 30356 tablet_replica.cc:442] T 5d758d71f9ac46bbbbfe544dcaf3bf50 P 24014678d6a345abaf05bbdb3d0003ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:26:02.438894 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.436041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41558 (local address 127.21.81.194:38535)
0809 19:26:02.436083 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:02.436085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:02.436093 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:26:02.436132 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:02.436133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:02.436154 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:02.436236 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:26:02.436408 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.437152 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.438049 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.438359 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.438434 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.438552 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:26:02.438680 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:26:02.438754 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:02.438803 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 19:26:02.438824 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:26:02.444193 30403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.441436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:42947 (local address 127.21.81.194:57355)
0809 19:26:02.441665 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:02.441681 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:02.441733 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:02.441849 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:02.441850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:02.441851 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:26:02.441911 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.441912 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.442763 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.442764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.443340 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.443341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.443818 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.443819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.443866 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.443881 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:26:02.444045 (+ 164us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:26:02.444046 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:26:02.444080 (+ 34us) client_negotiation.cc:240] Negotiation successful
0809 19:26:02.444124 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
I20260809 19:26:02.444195 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.441510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:57355 (local address 127.21.81.193:42947)
0809 19:26:02.441588 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:02.441591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:02.441737 (+ 146us) server_negotiation.cc:407] Connection header received
0809 19:26:02.441769 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:02.441770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:02.441783 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:02.441848 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:26:02.441956 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.442673 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.443415 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.443716 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.443782 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.443952 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:26:02.443986 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:26:02.444040 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:02.444084 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 19:26:02.444108 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 19:26:02.447062 30271 raft_consensus.cc:1261] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Refusing update from remote peer 24014678d6a345abaf05bbdb3d0003ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:26:02.447104 29972 raft_consensus.cc:1261] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Refusing update from remote peer 24014678d6a345abaf05bbdb3d0003ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:26:02.447341 30356 consensus_queue.cc:1035] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:26:02.447486 30382 consensus_queue.cc:1035] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:26:02.449191 30367 mvcc.cc:205] Tried to move back new op lower bound from 7316699391780933632 to 7316699391192518656. Current Snapshot: MvccSnapshot[applied={T|T < 7316699391780933632}]
I20260809 19:26:02.456460 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.453720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:36545 (local address 127.21.81.194:38535)
0809 19:26:02.453765 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:02.453767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:02.453916 (+ 149us) server_negotiation.cc:407] Connection header received
0809 19:26:02.453987 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:02.453988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:02.453998 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:02.454049 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:26:02.454160 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.454645 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.455768 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.455982 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.456038 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.456223 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:26:02.456257 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:26:02.456306 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:02.456364 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 19:26:02.456384 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:26:02.456482 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.453676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:38535 (local address 127.21.81.193:36545)
0809 19:26:02.453838 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:02.453854 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:02.453928 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:02.454052 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:02.454053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:02.454054 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:26:02.454105 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.454107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.454735 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.454736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.455666 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:02.455669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.456076 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.456077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:02.456124 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.456139 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:26:02.456306 (+ 167us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:26:02.456309 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:26:02.456360 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:26:02.456405 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
I20260809 19:26:02.465994 29806 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:26:02.466110 29806 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:26:02.466782 29806 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a: Sending DeleteTablet for 3 replicas of tablet 698bd93ed6e149e29c0744409e828651
I20260809 19:26:02.467085 29952 tablet_service.cc:1514] Processing DeleteTablet for tablet 698bd93ed6e149e29c0744409e828651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:02 UTC) from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:44026
I20260809 19:26:02.467139 30250 tablet_service.cc:1514] Processing DeleteTablet for tablet 698bd93ed6e149e29c0744409e828651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:02 UTC) from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:42426
I20260809 19:26:02.467213 30374 tablet_replica.cc:330] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46: stopping tablet replica
I20260809 19:26:02.467237 30375 tablet_replica.cc:330] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b: stopping tablet replica
I20260809 19:26:02.467288 30375 raft_consensus.cc:2229] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:26:02.467283 30374 raft_consensus.cc:2229] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:26:02.467324 30375 raft_consensus.cc:2258] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:02.467329 30374 raft_consensus.cc:2258] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:02.467643 30375 ts_tablet_manager.cc:1905] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:26:02.467650 30099 tablet_service.cc:1514] Processing DeleteTablet for tablet 698bd93ed6e149e29c0744409e828651 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:02 UTC) from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:41552
I20260809 19:26:02.467768 30376 tablet_replica.cc:330] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca: stopping tablet replica
I20260809 19:26:02.467835 30376 raft_consensus.cc:2229] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:26:02.467924 30376 raft_consensus.cc:2258] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:02.469421 30375 ts_tablet_manager.cc:1918] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:26:02.469475 30375 log.cc:1196] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/698bd93ed6e149e29c0744409e828651
I20260809 19:26:02.469731 30375 ts_tablet_manager.cc:1939] T 698bd93ed6e149e29c0744409e828651 P 83081c62f0e9449f909154f45694a11b: Deleting consensus metadata
I20260809 19:26:02.470145 30376 ts_tablet_manager.cc:1905] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:26:02.470826 30374 ts_tablet_manager.cc:1905] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:26:02.471741 30376 ts_tablet_manager.cc:1918] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:26:02.471807 30376 log.cc:1196] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/698bd93ed6e149e29c0744409e828651
I20260809 19:26:02.472023 30376 ts_tablet_manager.cc:1939] T 698bd93ed6e149e29c0744409e828651 P 24014678d6a345abaf05bbdb3d0003ca: Deleting consensus metadata
I20260809 19:26:02.472368 29791 catalog_manager.cc:4841] TS 24014678d6a345abaf05bbdb3d0003ca (127.21.81.194:38535): tablet 698bd93ed6e149e29c0744409e828651 (table test-table [id=1c17d1b21bdb4cfab0b5a7105ed2d433]) successfully deleted
I20260809 19:26:02.472513 30374 ts_tablet_manager.cc:1918] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:26:02.472572 30374 log.cc:1196] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/wal/wals/698bd93ed6e149e29c0744409e828651
I20260809 19:26:02.472715 29793 catalog_manager.cc:4841] TS 83081c62f0e9449f909154f45694a11b (127.21.81.195:44731): tablet 698bd93ed6e149e29c0744409e828651 (table test-table [id=1c17d1b21bdb4cfab0b5a7105ed2d433]) successfully deleted
I20260809 19:26:02.472795 30374 ts_tablet_manager.cc:1939] T 698bd93ed6e149e29c0744409e828651 P 30a6da08dd544f4c8a4206925dd4fa46: Deleting consensus metadata
I20260809 19:26:02.473126 29792 catalog_manager.cc:4841] TS 30a6da08dd544f4c8a4206925dd4fa46 (127.21.81.193:42947): tablet 698bd93ed6e149e29c0744409e828651 (table test-table [id=1c17d1b21bdb4cfab0b5a7105ed2d433]) successfully deleted
Aug 09 19:26:02 dist-test-slave-njxd krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 19:26:02 dist-test-slave-njxd krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:02 dist-test-slave-njxd krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 19:26:02.501324 29858 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:02.493897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47944 (local address 127.21.81.254:34111)
0809 19:26:02.493948 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:02.493951 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:02.494166 (+ 215us) server_negotiation.cc:407] Connection header received
0809 19:26:02.494251 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:02.494253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:02.494266 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:02.494336 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:02.495124 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.495942 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.497052 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.497380 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.497455 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.501252 (+ 3797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47944: BlockingRecv error: recv got EOF from 127.0.0.1:47944 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 19:26:02 dist-test-slave-njxd krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM
I20260809 19:26:02.511421 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.502677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47952 (local address 127.21.81.254:34111)
0809 19:26:02.502720 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:02.502722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:02.502953 (+ 231us) server_negotiation.cc:407] Connection header received
0809 19:26:02.503008 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:02.503009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:02.503019 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:02.503078 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:02.503639 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.504447 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.505458 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.505828 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.505902 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.508506 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:02.508508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:02.508508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:02.508516 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:02.510204 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:02.510601 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:02.510602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:02.510603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:02.510667 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:02.510877 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:02.510878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:02.510878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:02.511024 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:02.511117 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:02.511249 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 19:26:02.511326 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:26:02.513149 29806 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:26:02.513338 29806 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:26:02.514850 30250 tablet_service.cc:1467] Processing CreateTablet for tablet 382c1a17402d42e094ce8537217d1288 (DEFAULT_TABLE table=test-table [id=7a9e1ab49de04b7da17df26b56bb2003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:26:02.514842 30099 tablet_service.cc:1467] Processing CreateTablet for tablet 382c1a17402d42e094ce8537217d1288 (DEFAULT_TABLE table=test-table [id=7a9e1ab49de04b7da17df26b56bb2003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:26:02.514989 30099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 382c1a17402d42e094ce8537217d1288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:02.514988 30250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 382c1a17402d42e094ce8537217d1288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:02.516367 30350 tablet_bootstrap.cc:492] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca: Bootstrap starting.
I20260809 19:26:02.516376 30353 tablet_bootstrap.cc:492] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b: Bootstrap starting.
I20260809 19:26:02.516407 29952 tablet_service.cc:1467] Processing CreateTablet for tablet 382c1a17402d42e094ce8537217d1288 (DEFAULT_TABLE table=test-table [id=7a9e1ab49de04b7da17df26b56bb2003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:26:02.516527 29952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 382c1a17402d42e094ce8537217d1288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:02.516832 30353 tablet_bootstrap.cc:654] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:02.516842 30350 tablet_bootstrap.cc:654] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:02.518050 30351 tablet_bootstrap.cc:492] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46: Bootstrap starting.
I20260809 19:26:02.518396 30351 tablet_bootstrap.cc:654] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:02.518898 30350 tablet_bootstrap.cc:492] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca: No bootstrap required, opened a new log
I20260809 19:26:02.518956 30350 ts_tablet_manager.cc:1397] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 19:26:02.519088 30350 raft_consensus.cc:348] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.519105 30353 tablet_bootstrap.cc:492] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b: No bootstrap required, opened a new log
I20260809 19:26:02.519140 30350 raft_consensus.cc:374] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:02.519155 30350 raft_consensus.cc:729] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24014678d6a345abaf05bbdb3d0003ca, State: Initialized, Role: FOLLOWER
I20260809 19:26:02.519161 30353 ts_tablet_manager.cc:1397] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 19:26:02.519217 30350 consensus_queue.cc:260] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.519323 30350 ts_tablet_manager.cc:1428] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:02.519325 30353 raft_consensus.cc:348] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.519390 30353 raft_consensus.cc:374] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:02.519409 30353 raft_consensus.cc:729] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83081c62f0e9449f909154f45694a11b, State: Initialized, Role: FOLLOWER
I20260809 19:26:02.519454 30353 consensus_queue.cc:260] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.519538 30353 ts_tablet_manager.cc:1428] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 19:26:02.522063 30351 tablet_bootstrap.cc:492] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46: No bootstrap required, opened a new log
I20260809 19:26:02.522130 30351 ts_tablet_manager.cc:1397] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 19:26:02.522238 30351 raft_consensus.cc:348] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.522306 30351 raft_consensus.cc:374] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:02.522326 30351 raft_consensus.cc:729] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a6da08dd544f4c8a4206925dd4fa46, State: Initialized, Role: FOLLOWER
I20260809 19:26:02.522368 30351 consensus_queue.cc:260] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.522465 30351 ts_tablet_manager.cc:1428] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:02.725292 30407 raft_consensus.cc:482] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:26:02.725405 30407 raft_consensus.cc:504] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.725627 30407 leader_election.cc:288] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30a6da08dd544f4c8a4206925dd4fa46 (127.21.81.193:42947), 83081c62f0e9449f909154f45694a11b (127.21.81.195:44731)
I20260809 19:26:02.725878 29972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382c1a17402d42e094ce8537217d1288" candidate_uuid: "24014678d6a345abaf05bbdb3d0003ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30a6da08dd544f4c8a4206925dd4fa46" is_pre_election: true
I20260809 19:26:02.725898 30271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382c1a17402d42e094ce8537217d1288" candidate_uuid: "24014678d6a345abaf05bbdb3d0003ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83081c62f0e9449f909154f45694a11b" is_pre_election: true
I20260809 19:26:02.725996 30271 raft_consensus.cc:2454] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24014678d6a345abaf05bbdb3d0003ca in term 0.
I20260809 19:26:02.726001 29972 raft_consensus.cc:2454] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24014678d6a345abaf05bbdb3d0003ca in term 0.
I20260809 19:26:02.726189 30051 leader_election.cc:302] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24014678d6a345abaf05bbdb3d0003ca, 83081c62f0e9449f909154f45694a11b; no voters:
I20260809 19:26:02.726300 30407 raft_consensus.cc:2781] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:26:02.726368 30407 raft_consensus.cc:482] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:26:02.726399 30407 raft_consensus.cc:3037] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:02.727180 30407 raft_consensus.cc:504] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.727329 30407 leader_election.cc:288] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [CANDIDATE]: Term 1 election: Requested vote from peers 30a6da08dd544f4c8a4206925dd4fa46 (127.21.81.193:42947), 83081c62f0e9449f909154f45694a11b (127.21.81.195:44731)
I20260809 19:26:02.727471 29972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382c1a17402d42e094ce8537217d1288" candidate_uuid: "24014678d6a345abaf05bbdb3d0003ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30a6da08dd544f4c8a4206925dd4fa46"
I20260809 19:26:02.727500 30271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382c1a17402d42e094ce8537217d1288" candidate_uuid: "24014678d6a345abaf05bbdb3d0003ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83081c62f0e9449f909154f45694a11b"
I20260809 19:26:02.727556 29972 raft_consensus.cc:3037] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:02.727577 30271 raft_consensus.cc:3037] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:02.728379 30271 raft_consensus.cc:2454] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24014678d6a345abaf05bbdb3d0003ca in term 1.
I20260809 19:26:02.728382 29972 raft_consensus.cc:2454] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24014678d6a345abaf05bbdb3d0003ca in term 1.
I20260809 19:26:02.728597 30051 leader_election.cc:302] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24014678d6a345abaf05bbdb3d0003ca, 83081c62f0e9449f909154f45694a11b; no voters:
I20260809 19:26:02.728689 30407 raft_consensus.cc:2781] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:26:02.728749 30407 raft_consensus.cc:686] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [term 1 LEADER]: Becoming Leader. State: Replica: 24014678d6a345abaf05bbdb3d0003ca, State: Running, Role: LEADER
I20260809 19:26:02.728801 30407 consensus_queue.cc:237] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.729406 29807 catalog_manager.cc:5495] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 24014678d6a345abaf05bbdb3d0003ca (127.21.81.194). New cstate: current_term: 1 leader_uuid: "24014678d6a345abaf05bbdb3d0003ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } health_report { overall_health: UNKNOWN } } }
I20260809 19:26:02.747520 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:02.745088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41562 (local address 127.21.81.194:38535)
0809 19:26:02.745158 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:02.745161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:02.745214 (+ 53us) server_negotiation.cc:407] Connection header received
0809 19:26:02.745305 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:02.745305 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:02.745320 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:02.745388 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:26:02.745535 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.746052 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.746815 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:02.747018 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:02.747106 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:02.747249 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:26:02.747290 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:26:02.747346 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:02.747409 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 19:26:02.747433 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 19:26:02.748451 29972 raft_consensus.cc:1261] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Refusing update from remote peer 24014678d6a345abaf05bbdb3d0003ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:26:02.748451 30271 raft_consensus.cc:1261] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Refusing update from remote peer 24014678d6a345abaf05bbdb3d0003ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:26:02.748700 30382 consensus_queue.cc:1035] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:26:02.748806 30407 consensus_queue.cc:1035] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:26:02.750219 30366 mvcc.cc:205] Tried to move back new op lower bound from 7316699393015767040 to 7316699392937353216. Current Snapshot: MvccSnapshot[applied={T|T < 7316699393015767040 or (T in {7316699393015767040})}]
I20260809 19:26:02.751499 29807 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:26:02.751577 29807 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:26:02.752229 29807 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a: Sending DeleteTablet for 3 replicas of tablet 382c1a17402d42e094ce8537217d1288
I20260809 19:26:02.752466 29952 tablet_service.cc:1514] Processing DeleteTablet for tablet 382c1a17402d42e094ce8537217d1288 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:02 UTC) from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:44026
I20260809 19:26:02.752494 30099 tablet_service.cc:1514] Processing DeleteTablet for tablet 382c1a17402d42e094ce8537217d1288 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:02 UTC) from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:41552
I20260809 19:26:02.752526 30250 tablet_service.cc:1514] Processing DeleteTablet for tablet 382c1a17402d42e094ce8537217d1288 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:02 UTC) from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:42426
I20260809 19:26:02.752586 30374 tablet_replica.cc:330] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46: stopping tablet replica
I20260809 19:26:02.752616 30375 tablet_replica.cc:330] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b: stopping tablet replica
I20260809 19:26:02.752650 30374 raft_consensus.cc:2229] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:26:02.752635 30376 tablet_replica.cc:330] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca: stopping tablet replica
I20260809 19:26:02.752681 30375 raft_consensus.cc:2229] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:26:02.752691 30374 raft_consensus.cc:2258] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:02.752707 30376 raft_consensus.cc:2229] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:26:02.752744 30375 raft_consensus.cc:2258] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:02.752767 30376 raft_consensus.cc:2258] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:02.753032 30376 ts_tablet_manager.cc:1905] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:26:02.753147 30374 ts_tablet_manager.cc:1905] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:26:02.754324 29807 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 19:26:02.754472 30375 ts_tablet_manager.cc:1905] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:26:02.754508 30374 ts_tablet_manager.cc:1918] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:26:02.754551 30374 log.cc:1196] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-0/wal/wals/382c1a17402d42e094ce8537217d1288
W20260809 19:26:02.754539 29807 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:26:02.754675 30376 ts_tablet_manager.cc:1918] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:26:02.754727 30376 log.cc:1196] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/382c1a17402d42e094ce8537217d1288
I20260809 19:26:02.754812 30374 ts_tablet_manager.cc:1939] T 382c1a17402d42e094ce8537217d1288 P 30a6da08dd544f4c8a4206925dd4fa46: Deleting consensus metadata
I20260809 19:26:02.754933 30376 ts_tablet_manager.cc:1939] T 382c1a17402d42e094ce8537217d1288 P 24014678d6a345abaf05bbdb3d0003ca: Deleting consensus metadata
I20260809 19:26:02.755141 29792 catalog_manager.cc:4841] TS 30a6da08dd544f4c8a4206925dd4fa46 (127.21.81.193:42947): tablet 382c1a17402d42e094ce8537217d1288 (table test-table [id=7a9e1ab49de04b7da17df26b56bb2003]) successfully deleted
I20260809 19:26:02.755242 29791 catalog_manager.cc:4841] TS 24014678d6a345abaf05bbdb3d0003ca (127.21.81.194:38535): tablet 382c1a17402d42e094ce8537217d1288 (table test-table [id=7a9e1ab49de04b7da17df26b56bb2003]) successfully deleted
I20260809 19:26:02.755815 30099 tablet_service.cc:1467] Processing CreateTablet for tablet 30d950f3ffaf4e3c8a0698a874d6a873 (DEFAULT_TABLE table=test-table [id=d6cf689f72b34141940033d60412ee8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:26:02.755832 29952 tablet_service.cc:1467] Processing CreateTablet for tablet 30d950f3ffaf4e3c8a0698a874d6a873 (DEFAULT_TABLE table=test-table [id=d6cf689f72b34141940033d60412ee8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:26:02.755920 30099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d950f3ffaf4e3c8a0698a874d6a873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:02.755939 29952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d950f3ffaf4e3c8a0698a874d6a873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:02.756119 30375 ts_tablet_manager.cc:1918] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:26:02.756167 30375 log.cc:1196] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/382c1a17402d42e094ce8537217d1288
I20260809 19:26:02.756376 30375 ts_tablet_manager.cc:1939] T 382c1a17402d42e094ce8537217d1288 P 83081c62f0e9449f909154f45694a11b: Deleting consensus metadata
I20260809 19:26:02.756527 30250 tablet_service.cc:1467] Processing CreateTablet for tablet 30d950f3ffaf4e3c8a0698a874d6a873 (DEFAULT_TABLE table=test-table [id=d6cf689f72b34141940033d60412ee8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:26:02.756618 30250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30d950f3ffaf4e3c8a0698a874d6a873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:02.756778 30350 tablet_bootstrap.cc:492] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca: Bootstrap starting.
I20260809 19:26:02.757078 30350 tablet_bootstrap.cc:654] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:02.757598 29793 catalog_manager.cc:4841] TS 83081c62f0e9449f909154f45694a11b (127.21.81.195:44731): tablet 382c1a17402d42e094ce8537217d1288 (table test-table [id=7a9e1ab49de04b7da17df26b56bb2003]) successfully deleted
I20260809 19:26:02.757656 30351 tablet_bootstrap.cc:492] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46: Bootstrap starting.
I20260809 19:26:02.757876 30353 tablet_bootstrap.cc:492] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b: Bootstrap starting.
I20260809 19:26:02.757948 30351 tablet_bootstrap.cc:654] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:02.758514 30353 tablet_bootstrap.cc:654] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:02.758531 30351 tablet_bootstrap.cc:492] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46: No bootstrap required, opened a new log
I20260809 19:26:02.758574 30351 ts_tablet_manager.cc:1397] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:26:02.758706 30350 tablet_bootstrap.cc:492] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca: No bootstrap required, opened a new log
I20260809 19:26:02.758709 30351 raft_consensus.cc:348] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.758767 30350 ts_tablet_manager.cc:1397] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:26:02.758795 30351 raft_consensus.cc:374] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:02.758831 30351 raft_consensus.cc:729] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a6da08dd544f4c8a4206925dd4fa46, State: Initialized, Role: FOLLOWER
I20260809 19:26:02.758873 30350 raft_consensus.cc:348] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.758885 30351 consensus_queue.cc:260] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.758971 30350 raft_consensus.cc:374] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:02.759001 30350 raft_consensus.cc:729] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24014678d6a345abaf05bbdb3d0003ca, State: Initialized, Role: FOLLOWER
I20260809 19:26:02.759006 30351 ts_tablet_manager.cc:1428] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:26:02.759052 30350 consensus_queue.cc:260] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.759152 30350 ts_tablet_manager.cc:1428] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:26:02.762676 30353 tablet_bootstrap.cc:492] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b: No bootstrap required, opened a new log
I20260809 19:26:02.762740 30353 ts_tablet_manager.cc:1397] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260809 19:26:02.762904 30353 raft_consensus.cc:348] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.762976 30353 raft_consensus.cc:374] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:02.763010 30353 raft_consensus.cc:729] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83081c62f0e9449f909154f45694a11b, State: Initialized, Role: FOLLOWER
I20260809 19:26:02.763062 30353 consensus_queue.cc:260] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.763176 30353 ts_tablet_manager.cc:1428] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:02.994719 30356 raft_consensus.cc:482] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:26:02.994845 30356 raft_consensus.cc:504] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.995046 30356 leader_election.cc:288] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30a6da08dd544f4c8a4206925dd4fa46 (127.21.81.193:42947), 83081c62f0e9449f909154f45694a11b (127.21.81.195:44731)
I20260809 19:26:02.995275 30271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d950f3ffaf4e3c8a0698a874d6a873" candidate_uuid: "24014678d6a345abaf05bbdb3d0003ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83081c62f0e9449f909154f45694a11b" is_pre_election: true
I20260809 19:26:02.995275 29972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d950f3ffaf4e3c8a0698a874d6a873" candidate_uuid: "24014678d6a345abaf05bbdb3d0003ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30a6da08dd544f4c8a4206925dd4fa46" is_pre_election: true
I20260809 19:26:02.995383 30271 raft_consensus.cc:2454] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24014678d6a345abaf05bbdb3d0003ca in term 0.
I20260809 19:26:02.995383 29972 raft_consensus.cc:2454] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24014678d6a345abaf05bbdb3d0003ca in term 0.
I20260809 19:26:02.995553 30051 leader_election.cc:302] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24014678d6a345abaf05bbdb3d0003ca, 83081c62f0e9449f909154f45694a11b; no voters:
I20260809 19:26:02.995671 30356 raft_consensus.cc:2781] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:26:02.995714 30356 raft_consensus.cc:482] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:26:02.995736 30356 raft_consensus.cc:3037] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:02.996452 30356 raft_consensus.cc:504] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.996583 30356 leader_election.cc:288] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [CANDIDATE]: Term 1 election: Requested vote from peers 30a6da08dd544f4c8a4206925dd4fa46 (127.21.81.193:42947), 83081c62f0e9449f909154f45694a11b (127.21.81.195:44731)
I20260809 19:26:02.996730 30271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d950f3ffaf4e3c8a0698a874d6a873" candidate_uuid: "24014678d6a345abaf05bbdb3d0003ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83081c62f0e9449f909154f45694a11b"
I20260809 19:26:02.996744 29972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30d950f3ffaf4e3c8a0698a874d6a873" candidate_uuid: "24014678d6a345abaf05bbdb3d0003ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30a6da08dd544f4c8a4206925dd4fa46"
I20260809 19:26:02.996803 30271 raft_consensus.cc:3037] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:02.996811 29972 raft_consensus.cc:3037] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:02.997524 29972 raft_consensus.cc:2454] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24014678d6a345abaf05bbdb3d0003ca in term 1.
I20260809 19:26:02.997524 30271 raft_consensus.cc:2454] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24014678d6a345abaf05bbdb3d0003ca in term 1.
I20260809 19:26:02.997697 30051 leader_election.cc:302] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24014678d6a345abaf05bbdb3d0003ca, 83081c62f0e9449f909154f45694a11b; no voters:
I20260809 19:26:02.997798 30356 raft_consensus.cc:2781] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:26:02.997857 30356 raft_consensus.cc:686] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [term 1 LEADER]: Becoming Leader. State: Replica: 24014678d6a345abaf05bbdb3d0003ca, State: Running, Role: LEADER
I20260809 19:26:02.997913 30356 consensus_queue.cc:237] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } }
I20260809 19:26:02.998435 29807 catalog_manager.cc:5495] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 24014678d6a345abaf05bbdb3d0003ca (127.21.81.194). New cstate: current_term: 1 leader_uuid: "24014678d6a345abaf05bbdb3d0003ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24014678d6a345abaf05bbdb3d0003ca" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 } health_report { overall_health: UNKNOWN } } }
I20260809 19:26:03.045890 29972 raft_consensus.cc:1261] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Refusing update from remote peer 24014678d6a345abaf05bbdb3d0003ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:26:03.045882 30271 raft_consensus.cc:1261] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Refusing update from remote peer 24014678d6a345abaf05bbdb3d0003ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:26:03.046124 30356 consensus_queue.cc:1035] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "30a6da08dd544f4c8a4206925dd4fa46" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:26:03.046231 30407 consensus_queue.cc:1035] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "83081c62f0e9449f909154f45694a11b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:26:03.055819 29807 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:26:03.055912 29807 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:26:03.056551 29807 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 614bb27e14f14f1fbe5f3c0b0b43b91a: Sending DeleteTablet for 3 replicas of tablet 30d950f3ffaf4e3c8a0698a874d6a873
I20260809 19:26:03.056779 29952 tablet_service.cc:1514] Processing DeleteTablet for tablet 30d950f3ffaf4e3c8a0698a874d6a873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:03 UTC) from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:44026
I20260809 19:26:03.056898 30374 tablet_replica.cc:330] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46: stopping tablet replica
I20260809 19:26:03.056885 30099 tablet_service.cc:1514] Processing DeleteTablet for tablet 30d950f3ffaf4e3c8a0698a874d6a873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:03 UTC) from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:41552
I20260809 19:26:03.056906 30250 tablet_service.cc:1514] Processing DeleteTablet for tablet 30d950f3ffaf4e3c8a0698a874d6a873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:03 UTC) from {username='oryx', principal='oryx/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:42426
I20260809 19:26:03.056953 30374 raft_consensus.cc:2229] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:26:03.056988 30375 tablet_replica.cc:330] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b: stopping tablet replica
I20260809 19:26:03.056996 30374 raft_consensus.cc:2258] T 30d950f3ffaf4e3c8a0698a874d6a873 P 30a6da08dd544f4c8a4206925dd4fa46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:03.057031 30375 raft_consensus.cc:2229] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:26:03.057037 30376 tablet_replica.cc:330] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca: stopping tablet replica
I20260809 19:26:03.057073 30375 raft_consensus.cc:2258] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:03.057090 30376 raft_consensus.cc:2229] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:26:03.057144 30376 raft_consensus.cc:2258] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:03.057265 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 29883
I20260809 19:26:03.057444 30375 ts_tablet_manager.cc:1905] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:26:03.057472 30376 ts_tablet_manager.cc:1905] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:26:03.059285 30376 ts_tablet_manager.cc:1918] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:26:03.059340 30376 log.cc:1196] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/30d950f3ffaf4e3c8a0698a874d6a873
I20260809 19:26:03.059540 30376 ts_tablet_manager.cc:1939] T 30d950f3ffaf4e3c8a0698a874d6a873 P 24014678d6a345abaf05bbdb3d0003ca: Deleting consensus metadata
I20260809 19:26:03.059978 29791 catalog_manager.cc:4841] TS 24014678d6a345abaf05bbdb3d0003ca (127.21.81.194:38535): tablet 30d950f3ffaf4e3c8a0698a874d6a873 (table test-table [id=d6cf689f72b34141940033d60412ee8c]) successfully deleted
I20260809 19:26:03.060179 30375 ts_tablet_manager.cc:1918] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:26:03.060252 30375 log.cc:1196] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/30d950f3ffaf4e3c8a0698a874d6a873
I20260809 19:26:03.060457 30375 ts_tablet_manager.cc:1939] T 30d950f3ffaf4e3c8a0698a874d6a873 P 83081c62f0e9449f909154f45694a11b: Deleting consensus metadata
I20260809 19:26:03.060770 29793 catalog_manager.cc:4841] TS 83081c62f0e9449f909154f45694a11b (127.21.81.195:44731): tablet 30d950f3ffaf4e3c8a0698a874d6a873 (table test-table [id=d6cf689f72b34141940033d60412ee8c]) successfully deleted
W20260809 19:26:03.066728 29792 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:42947 (error 108)
I20260809 19:26:03.066985 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 30031
I20260809 19:26:03.067850 30445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:03.067089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:42947 (local address 127.0.0.1:44060)
0809 19:26:03.067756 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:03.067801 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:42947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":42,"threads_started":1}
W20260809 19:26:03.067970 29792 catalog_manager.cc:4584] TS 30a6da08dd544f4c8a4206925dd4fa46 (127.21.81.193:42947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 30d950f3ffaf4e3c8a0698a874d6a873: Network error: Client connection negotiation failed: client connection to 127.21.81.193:42947: connect: Connection refused (error 111)
I20260809 19:26:03.078328 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 30180
I20260809 19:26:03.084000 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 29773
2026-08-09T19:26:03Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3042 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:26:03 dist-test-slave-njxd krb5kdc[30449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:26:03 dist-test-slave-njxd krb5kdc[30449](info): set up 2 sockets
Aug 09 19:26:03 dist-test-slave-njxd krb5kdc[30449](info): commencing operation
krb5kdc: starting...
W20260809 19:26:04.141186 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:26:04 dist-test-slave-njxd krb5kdc[30449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:26:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:26:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:26:04.262638 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:45941
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:42249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:45941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:26:04.345110 30462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:04.345355 30462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:04.345393 30462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:04.346580 30462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:26:04.346644 30462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:04.346668 30462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:04.346690 30462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:26:04.346719 30462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:26:04.348174 30462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:42249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:45941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:45941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:04.348450 30462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:04.348682 30462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:04.353430 30468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:04.353437 30469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:04.353435 30471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:04.353693 30462 server_base.cc:1034] running on GCE node
I20260809 19:26:04.353919 30462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:04.354260 30462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:04.355427 30462 hybrid_clock.cc:648] HybridClock initialized: now 1786303564355416 us; error 34 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T19:26:04Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1274 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:26:04 dist-test-slave-njxd krb5kdc[30476](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:26:04 dist-test-slave-njxd krb5kdc[30476](info): set up 2 sockets
Aug 09 19:26:04 dist-test-slave-njxd krb5kdc[30476](info): commencing operation
krb5kdc: starting...
W20260809 19:26:05.438623 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:26:05 dist-test-slave-njxd krb5kdc[30476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:26:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:26:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:26:05.564139 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:32983
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:33277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:32983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:26:05.647636 30490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:05.647886 30490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:05.647924 30490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:05.649137 30490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:26:05.649201 30490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:05.649230 30490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:05.649260 30490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:26:05.649284 30490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:26:05.650866 30490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:33277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:32983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:32983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:05.651149 30490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:05.651376 30490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:05.656260 30499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:05.656251 30496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:05.656351 30490 server_base.cc:1034] running on GCE node
W20260809 19:26:05.656251 30497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:05.656718 30490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:05.657011 30490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:05.658156 30490 hybrid_clock.cc:648] HybridClock initialized: now 1786303565658145 us; error 34 us; skew 500 ppm
Aug 09 19:26:05 dist-test-slave-njxd krb5kdc[30476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:05.660681 30490 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:26:05.661154 30490 webserver.cc:460] Webserver started at http://127.21.81.254:39569/ using document root <none> and password file <none>
I20260809 19:26:05.661378 30490 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:05.661442 30490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:05.661629 30490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:05.662716 30490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "ca167b36177c435f83dc1149f8d5e8b9"
format_stamp: "Formatted at 2026-08-09 19:26:05 on dist-test-slave-njxd"
server_key: "52a489ba04d89394db909480c0c88c8d"
server_key_iv: "a4af95458a40fbc9acc1e8a74fa593ab"
server_key_version: "encryptionkey@0"
I20260809 19:26:05.663077 30490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "ca167b36177c435f83dc1149f8d5e8b9"
format_stamp: "Formatted at 2026-08-09 19:26:05 on dist-test-slave-njxd"
server_key: "52a489ba04d89394db909480c0c88c8d"
server_key_iv: "a4af95458a40fbc9acc1e8a74fa593ab"
server_key_version: "encryptionkey@0"
I20260809 19:26:05.664430 30490 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:26:05.665163 30506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:05.665330 30490 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:26:05.665428 30490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "ca167b36177c435f83dc1149f8d5e8b9"
format_stamp: "Formatted at 2026-08-09 19:26:05 on dist-test-slave-njxd"
server_key: "52a489ba04d89394db909480c0c88c8d"
server_key_iv: "a4af95458a40fbc9acc1e8a74fa593ab"
server_key_version: "encryptionkey@0"
I20260809 19:26:05.665508 30490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:05.699160 30490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:05.701725 30490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:05.701906 30490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:05.706084 30490 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:32983
I20260809 19:26:05.706126 30560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:32983 every 8 connection(s)
I20260809 19:26:05.706523 30490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:26:05.707437 30561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:05.709776 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 30490
I20260809 19:26:05.709908 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:26:05.710071 30561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9: Bootstrap starting.
I20260809 19:26:05.710136 21831 external_mini_cluster.cc:1428] Setting key 788ea3902ef2b9bef1babeaaeae2a6a7
I20260809 19:26:05.710836 30561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:05.711162 30561 log.cc:824] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9: Log is configured to *not* fsync() on all Append() calls
I20260809 19:26:05.711989 30561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9: No bootstrap required, opened a new log
I20260809 19:26:05.714224 30561 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca167b36177c435f83dc1149f8d5e8b9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32983 } }
I20260809 19:26:05.714403 30561 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:05.714442 30561 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca167b36177c435f83dc1149f8d5e8b9, State: Initialized, Role: FOLLOWER
I20260809 19:26:05.714608 30561 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca167b36177c435f83dc1149f8d5e8b9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32983 } }
I20260809 19:26:05.714696 30561 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:26:05.714725 30561 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:26:05.714769 30561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:05.715514 30561 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca167b36177c435f83dc1149f8d5e8b9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32983 } }
I20260809 19:26:05.715634 30561 leader_election.cc:302] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca167b36177c435f83dc1149f8d5e8b9; no voters:
I20260809 19:26:05.715816 30561 leader_election.cc:288] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:26:05.715843 30567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:26:05.715967 30567 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 1 LEADER]: Becoming Leader. State: Replica: ca167b36177c435f83dc1149f8d5e8b9, State: Running, Role: LEADER
I20260809 19:26:05.716110 30561 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:26:05.716136 30567 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca167b36177c435f83dc1149f8d5e8b9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32983 } }
I20260809 19:26:05.716557 30568 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca167b36177c435f83dc1149f8d5e8b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca167b36177c435f83dc1149f8d5e8b9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32983 } } }
I20260809 19:26:05.716572 30569 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca167b36177c435f83dc1149f8d5e8b9. Latest consensus state: current_term: 1 leader_uuid: "ca167b36177c435f83dc1149f8d5e8b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca167b36177c435f83dc1149f8d5e8b9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32983 } } }
I20260809 19:26:05.716684 30568 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [sys.catalog]: This master's current role is: LEADER
I20260809 19:26:05.716706 30569 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [sys.catalog]: This master's current role is: LEADER
I20260809 19:26:05.716918 30572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:26:05.717543 30572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 19:26:05 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:26:05.720641 30572 catalog_manager.cc:1347] Generated new cluster ID: 35085fc1a6fb45659495b0a934ea78d0
I20260809 19:26:05.720692 30572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:26:05.722316 30565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:05.710677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49108 (local address 127.21.81.254:32983)
0809 19:26:05.710854 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:05.710862 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:05.710905 (+ 43us) server_negotiation.cc:407] Connection header received
0809 19:26:05.711451 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:05.711460 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:05.711524 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:05.711680 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:05.712469 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:05.713334 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:05.714525 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:05.715160 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:05.715277 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:05.718247 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:05.718251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:05.718251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:05.718263 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:05.720418 (+ 2155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:05.720844 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:05.720847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:05.720847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:05.720956 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:05.721174 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:05.721175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:05.721175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:05.721492 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:05.721658 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:05.721938 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 19:26:05.722081 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260809 19:26:05.738708 30572 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:26:05.739440 30572 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:26:05.742066 30572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9: Generated new TSK 0
I20260809 19:26:05.742249 30572 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:26:05.778542 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:32983
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:33277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:26:05.879770 30590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:05.880015 30590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:05.880052 30590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:05.881247 30590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:05.881309 30590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:05.881356 30590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:26:05.882820 30590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:33277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:32983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:05.883103 30590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:05.883325 30590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:05.888334 30599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:05.888362 30590 server_base.cc:1034] running on GCE node
W20260809 19:26:05.888345 30596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:05.888332 30597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:05.888863 30590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:05.889137 30590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:05.890271 30590 hybrid_clock.cc:648] HybridClock initialized: now 1786303565890262 us; error 28 us; skew 500 ppm
Aug 09 19:26:05 dist-test-slave-njxd krb5kdc[30476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:05.892871 30590 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:26:05.893257 30590 webserver.cc:460] Webserver started at http://127.21.81.193:33551/ using document root <none> and password file <none>
I20260809 19:26:05.893448 30590 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:05.893487 30590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:05.893626 30590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:05.894677 30590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "17007b4f301d40c3bce58b9eaeb353d1"
format_stamp: "Formatted at 2026-08-09 19:26:05 on dist-test-slave-njxd"
server_key: "681c26d1c7151224b465afdeb2ce1dfe"
server_key_iv: "60ed5f54f68b0993e5070d8c576794bd"
server_key_version: "encryptionkey@0"
I20260809 19:26:05.895041 30590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "17007b4f301d40c3bce58b9eaeb353d1"
format_stamp: "Formatted at 2026-08-09 19:26:05 on dist-test-slave-njxd"
server_key: "681c26d1c7151224b465afdeb2ce1dfe"
server_key_iv: "60ed5f54f68b0993e5070d8c576794bd"
server_key_version: "encryptionkey@0"
I20260809 19:26:05.896415 30590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:26:05.897102 30607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:05.897279 30590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:26:05.897362 30590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "17007b4f301d40c3bce58b9eaeb353d1"
format_stamp: "Formatted at 2026-08-09 19:26:05 on dist-test-slave-njxd"
server_key: "681c26d1c7151224b465afdeb2ce1dfe"
server_key_iv: "60ed5f54f68b0993e5070d8c576794bd"
server_key_version: "encryptionkey@0"
I20260809 19:26:05.897418 30590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:05.914294 30590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:05.916206 30590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:05.916323 30590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:05.916646 30590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:26:05.916963 30590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:05.916999 30590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:05.917021 30590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:05.917034 30590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:05.922979 30590 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:38589
I20260809 19:26:05.923089 30729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:38589 every 8 connection(s)
I20260809 19:26:05.923383 30590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:26:05.924796 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 30590
I20260809 19:26:05.924886 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:26:05.925079 21831 external_mini_cluster.cc:1428] Setting key 42360cfbed3f380e9e4f85f498e437d4
Aug 09 19:26:05 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:26:05.933210 30565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:05.924097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:45417 (local address 127.21.81.254:32983)
0809 19:26:05.924161 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:05.924163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:05.924402 (+ 239us) server_negotiation.cc:407] Connection header received
0809 19:26:05.924943 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:05.924944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:05.924958 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:05.925007 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:05.926093 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:05.926600 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:05.927854 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:05.928073 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:05.928138 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:05.930396 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:05.930398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:05.930398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:05.930406 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:05.931825 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:05.932238 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:05.932239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:05.932240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:05.932300 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:05.932553 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:05.932554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:05.932555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:05.932671 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:05.932739 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:05.933019 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 19:26:05.933116 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:26:05.933434 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:05.924062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:45417)
0809 19:26:05.924278 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:05.924320 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:05.924799 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:05.925088 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:05.925092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:05.925790 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:05.925980 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:05.925985 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:05.926707 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:05.926708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:05.927767 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:05.927771 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:05.928159 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:05.928160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:05.928322 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:05.928880 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:05.928888 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:05.930299 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:05.931907 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:05.931915 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:05.931918 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:05.932144 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:05.932374 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:05.932375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:05.932376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:05.932484 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:05.932742 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:05.932750 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:05.932889 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 19:26:05.932987 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 19:26:05.933147 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
I20260809 19:26:05.933969 30730 heartbeater.cc:347] Connected to a master server at 127.21.81.254:32983
I20260809 19:26:05.934075 30730 heartbeater.cc:464] Registering TS with master...
I20260809 19:26:05.934265 30730 heartbeater.cc:511] Master 127.21.81.254:32983 requested a full tablet report, sending...
I20260809 19:26:05.934801 30524 ts_manager.cc:194] Registered new tserver with Master: 17007b4f301d40c3bce58b9eaeb353d1 (127.21.81.193:38589)
I20260809 19:26:05.935429 30524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:45417
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:26:05.950491 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:32983
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:33277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:26:06.033950 30735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:06.034163 30735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:06.034188 30735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:06.035322 30735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:06.035385 30735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:06.035440 30735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:26:06.036885 30735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:33277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:32983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:06.037081 30735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:06.037256 30735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:06.042127 30741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:06.042176 30744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:06.042127 30742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:06.042322 30735 server_base.cc:1034] running on GCE node
I20260809 19:26:06.042629 30735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:06.042902 30735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:06.044061 30735 hybrid_clock.cc:648] HybridClock initialized: now 1786303566044055 us; error 35 us; skew 500 ppm
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303566, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:06.046627 30735 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:26:06.047060 30735 webserver.cc:460] Webserver started at http://127.21.81.194:40815/ using document root <none> and password file <none>
I20260809 19:26:06.047286 30735 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:06.047330 30735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:06.047484 30735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:06.048492 30735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "64b21a6dbb6547d9a8b30e22e111b2d4"
format_stamp: "Formatted at 2026-08-09 19:26:06 on dist-test-slave-njxd"
server_key: "cd372e07780a170c71f7e567fc068c26"
server_key_iv: "1dc9043e9bfa282701be0f29b697395f"
server_key_version: "encryptionkey@0"
I20260809 19:26:06.048854 30735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "64b21a6dbb6547d9a8b30e22e111b2d4"
format_stamp: "Formatted at 2026-08-09 19:26:06 on dist-test-slave-njxd"
server_key: "cd372e07780a170c71f7e567fc068c26"
server_key_iv: "1dc9043e9bfa282701be0f29b697395f"
server_key_version: "encryptionkey@0"
I20260809 19:26:06.050248 30735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:26:06.050982 30751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.051151 30735 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:26:06.051234 30735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "64b21a6dbb6547d9a8b30e22e111b2d4"
format_stamp: "Formatted at 2026-08-09 19:26:06 on dist-test-slave-njxd"
server_key: "cd372e07780a170c71f7e567fc068c26"
server_key_iv: "1dc9043e9bfa282701be0f29b697395f"
server_key_version: "encryptionkey@0"
I20260809 19:26:06.051287 30735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:06.067351 30735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:06.069996 30735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:06.070175 30735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:06.070499 30735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:26:06.070938 30735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:06.070998 30735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.071045 30735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:06.071075 30735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.079535 30735 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:34261
I20260809 19:26:06.079589 30871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:34261 every 8 connection(s)
I20260809 19:26:06.079910 30735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:26:06.086282 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 30735
I20260809 19:26:06.086407 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:26:06.086630 21831 external_mini_cluster.cc:1428] Setting key e71d042d52203d265bddcf4dd62ca60c
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303566, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:26:06.093799 30565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:06.080814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:39237 (local address 127.21.81.254:32983)
0809 19:26:06.080894 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:06.080896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:06.081120 (+ 224us) server_negotiation.cc:407] Connection header received
0809 19:26:06.081787 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:06.081789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:06.081804 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:06.081883 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:06.085036 (+ 3153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.085855 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.087438 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.087775 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.087848 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.090692 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:06.090694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:06.090694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:06.090701 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:06.092145 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:06.092681 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:06.092682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:06.092682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:06.092744 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:06.093066 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:06.093067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:06.093067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:06.093174 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:06.093228 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:06.093646 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 19:26:06.093729 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:26:06.094060 30874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:06.080781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:39237)
0809 19:26:06.080998 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:06.081041 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:06.081682 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:06.082020 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:06.082025 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:06.084583 (+ 2558us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:06.084924 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.084930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.085963 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.085965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.087299 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.087302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.087879 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.087880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.088087 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.088731 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:06.088743 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:06.090598 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:06.092227 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:06.092235 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:06.092237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:06.092579 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:06.092825 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:06.092826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:06.092826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:06.092991 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:06.093243 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:06.093250 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:06.093429 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 19:26:06.093568 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 19:26:06.093799 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":104,"threads_started":1}
I20260809 19:26:06.094609 30872 heartbeater.cc:347] Connected to a master server at 127.21.81.254:32983
I20260809 19:26:06.094728 30872 heartbeater.cc:464] Registering TS with master...
I20260809 19:26:06.094941 30872 heartbeater.cc:511] Master 127.21.81.254:32983 requested a full tablet report, sending...
I20260809 19:26:06.095319 30524 ts_manager.cc:194] Registered new tserver with Master: 64b21a6dbb6547d9a8b30e22e111b2d4 (127.21.81.194:34261)
I20260809 19:26:06.095706 30524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:39237
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:26:06.120494 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:32983
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:33277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:26:06.201489 30877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:06.201715 30877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:06.201738 30877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:06.202841 30877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:06.202877 30877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:06.202903 30877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:26:06.204339 30877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:33277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:32983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:06.204572 30877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:06.204849 30877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:06.209761 30886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:06.209776 30884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:06.209774 30882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:06.209944 30877 server_base.cc:1034] running on GCE node
I20260809 19:26:06.210249 30877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:06.210541 30877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:06.211699 30877 hybrid_clock.cc:648] HybridClock initialized: now 1786303566211684 us; error 40 us; skew 500 ppm
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303566, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:06.214227 30877 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:26:06.214676 30877 webserver.cc:460] Webserver started at http://127.21.81.195:35117/ using document root <none> and password file <none>
I20260809 19:26:06.214917 30877 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:06.214985 30877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:06.215132 30877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:06.216252 30877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "33fb0499865840469df2081c305215c5"
format_stamp: "Formatted at 2026-08-09 19:26:06 on dist-test-slave-njxd"
server_key: "cfbfa9792d8cefb264b8817aeff94c59"
server_key_iv: "b900046e175a023dc3ac41cad487e4e1"
server_key_version: "encryptionkey@0"
I20260809 19:26:06.216656 30877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "33fb0499865840469df2081c305215c5"
format_stamp: "Formatted at 2026-08-09 19:26:06 on dist-test-slave-njxd"
server_key: "cfbfa9792d8cefb264b8817aeff94c59"
server_key_iv: "b900046e175a023dc3ac41cad487e4e1"
server_key_version: "encryptionkey@0"
I20260809 19:26:06.218257 30877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:26:06.218956 30893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.219146 30877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:26:06.219219 30877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "33fb0499865840469df2081c305215c5"
format_stamp: "Formatted at 2026-08-09 19:26:06 on dist-test-slave-njxd"
server_key: "cfbfa9792d8cefb264b8817aeff94c59"
server_key_iv: "b900046e175a023dc3ac41cad487e4e1"
server_key_version: "encryptionkey@0"
I20260809 19:26:06.219269 30877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:06.229159 30877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:06.230998 30877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:06.231107 30877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:06.231294 30877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:26:06.231537 30877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:06.231567 30877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.231590 30877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:06.231608 30877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.237560 30877 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:32855
I20260809 19:26:06.237659 31015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:32855 every 8 connection(s)
I20260809 19:26:06.238005 30877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303566, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:26:06.245395 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 30877
I20260809 19:26:06.245529 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:26:06.245740 21831 external_mini_cluster.cc:1428] Setting key e595835307a6c5984e92ab50c5d36673
I20260809 19:26:06.247885 31018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:06.238771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:53781)
0809 19:26:06.238952 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:06.238997 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:06.239473 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:06.239693 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:06.239697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:06.240268 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:06.240467 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.240472 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.241130 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.241130 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.242218 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.242221 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.242616 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.242617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.242719 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.243263 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:06.243277 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:06.244726 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:06.246514 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:06.246520 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:06.246522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:06.246768 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:06.247018 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:06.247019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:06.247019 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:06.247117 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:06.247389 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:06.247393 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:06.247505 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 19:26:06.247586 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 19:26:06.247720 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":98,"threads_started":1}
I20260809 19:26:06.247900 30565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:06.238821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:53781 (local address 127.21.81.254:32983)
0809 19:26:06.238871 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:06.238873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:06.239086 (+ 213us) server_negotiation.cc:407] Connection header received
0809 19:26:06.239554 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:06.239555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:06.239565 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:06.239640 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:06.240545 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.241042 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.242293 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.242533 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.242607 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.244811 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:06.244813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:06.244813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:06.244820 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:06.246443 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:06.246868 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:06.246869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:06.246870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:06.246954 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:06.247172 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:06.247173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:06.247174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:06.247318 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:06.247391 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:06.247610 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 19:26:06.247684 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:26:06.248329 31016 heartbeater.cc:347] Connected to a master server at 127.21.81.254:32983
I20260809 19:26:06.248565 31016 heartbeater.cc:464] Registering TS with master...
I20260809 19:26:06.248764 31016 heartbeater.cc:511] Master 127.21.81.254:32983 requested a full tablet report, sending...
I20260809 19:26:06.249171 30524 ts_manager.cc:194] Registered new tserver with Master: 33fb0499865840469df2081c305215c5 (127.21.81.195:32855)
I20260809 19:26:06.249513 30524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:53781
I20260809 19:26:06.257762 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.193.keytab.
Entry for principal oryx/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.193.keytab.
WARNING: no policy specified for oryx/127.21.81.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.194.keytab.
Entry for principal oryx/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.194.keytab.
WARNING: no policy specified for oryx/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.195.keytab.
Entry for principal oryx/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.195.keytab.
I20260809 19:26:06.327965 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 30590
I20260809 19:26:06.334141 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 30735
I20260809 19:26:06.339686 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 30877
I20260809 19:26:06.345326 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 30490
I20260809 19:26:06.352439 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:32983
--webserver_interface=127.21.81.254
--webserver_port=39569
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:33277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:32983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:26:06.434700 31027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:06.434909 31027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:06.434932 31027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:06.436102 31027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:26:06.436146 31027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:06.436159 31027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:06.436172 31027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:26:06.436223 31027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:26:06.437788 31027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:33277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:32983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:32983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=39569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:06.438047 31027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:06.438280 31027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:06.443105 31032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:06.443146 31033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:06.443105 31036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:06.443467 31027 server_base.cc:1034] running on GCE node
I20260809 19:26:06.443774 31027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:06.444075 31027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:06.445238 31027 hybrid_clock.cc:648] HybridClock initialized: now 1786303566445229 us; error 35 us; skew 500 ppm
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303566, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:06.447929 31027 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:26:06.448318 31027 webserver.cc:460] Webserver started at http://127.21.81.254:39569/ using document root <none> and password file <none>
I20260809 19:26:06.448506 31027 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:06.448558 31027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:06.449889 31027 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:26:06.450500 31043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.450678 31027 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:26:06.450769 31027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "ca167b36177c435f83dc1149f8d5e8b9"
format_stamp: "Formatted at 2026-08-09 19:26:05 on dist-test-slave-njxd"
server_key: "52a489ba04d89394db909480c0c88c8d"
server_key_iv: "a4af95458a40fbc9acc1e8a74fa593ab"
server_key_version: "encryptionkey@0"
I20260809 19:26:06.451083 31027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:06.472151 31027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:06.474388 31027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:06.474501 31027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:06.478382 31027 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:32983
I20260809 19:26:06.478438 31098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:32983 every 8 connection(s)
I20260809 19:26:06.478806 31027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:26:06.480304 31099 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:26:06.481024 31099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9: Bootstrap starting.
I20260809 19:26:06.487605 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 31027
I20260809 19:26:06.487937 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:38589
--local_ip_for_outbound_sockets=127.21.81.193
--tserver_master_addrs=127.21.81.254:32983
--webserver_port=33551
--webserver_interface=127.21.81.193
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:33277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.81.193
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:26:06.490702 31099 log.cc:824] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9: Log is configured to *not* fsync() on all Append() calls
I20260809 19:26:06.492347 31099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:26:06.492637 31099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9: Bootstrap complete.
I20260809 19:26:06.494994 31099 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca167b36177c435f83dc1149f8d5e8b9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32983 } }
I20260809 19:26:06.495237 31099 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca167b36177c435f83dc1149f8d5e8b9, State: Initialized, Role: FOLLOWER
I20260809 19:26:06.495430 31099 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca167b36177c435f83dc1149f8d5e8b9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32983 } }
I20260809 19:26:06.495529 31099 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:26:06.495584 31099 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:26:06.495640 31099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:26:06.496609 31099 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca167b36177c435f83dc1149f8d5e8b9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32983 } }
I20260809 19:26:06.496734 31099 leader_election.cc:302] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca167b36177c435f83dc1149f8d5e8b9; no voters:
I20260809 19:26:06.496937 31099 leader_election.cc:288] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 19:26:06.496973 31103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:26:06.497094 31103 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [term 2 LEADER]: Becoming Leader. State: Replica: ca167b36177c435f83dc1149f8d5e8b9, State: Running, Role: LEADER
I20260809 19:26:06.497241 31099 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:26:06.497262 31103 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca167b36177c435f83dc1149f8d5e8b9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32983 } }
I20260809 19:26:06.497676 31104 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ca167b36177c435f83dc1149f8d5e8b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca167b36177c435f83dc1149f8d5e8b9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32983 } } }
I20260809 19:26:06.497767 31104 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [sys.catalog]: This master's current role is: LEADER
I20260809 19:26:06.498126 31109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:26:06.498171 31105 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca167b36177c435f83dc1149f8d5e8b9. Latest consensus state: current_term: 2 leader_uuid: "ca167b36177c435f83dc1149f8d5e8b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca167b36177c435f83dc1149f8d5e8b9" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32983 } } }
I20260809 19:26:06.498239 31105 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9 [sys.catalog]: This master's current role is: LEADER
I20260809 19:26:06.498688 31109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:26:06.498844 31109 catalog_manager.cc:1259] Loaded cluster ID: 35085fc1a6fb45659495b0a934ea78d0
I20260809 19:26:06.499010 31109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:26:06.500317 31109 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:26:06.500485 31109 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ca167b36177c435f83dc1149f8d5e8b9: Loaded TSK: 0
I20260809 19:26:06.500681 31109 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 19:26:06.572793 31101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:06.573045 31101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:06.573086 31101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:06.574383 31101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:06.574429 31101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:06.574455 31101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:26:06.575930 31101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:33277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.193.keytab
--principal=oryx/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:38589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=33551
--tserver_master_addrs=127.21.81.254:32983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:06.576166 31101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:06.576412 31101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:06.581419 31127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:06.581506 31101 server_base.cc:1034] running on GCE node
W20260809 19:26:06.581441 31129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:06.581655 31126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:06.581916 31101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:06.582199 31101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:06.583386 31101 hybrid_clock.cc:648] HybridClock initialized: now 1786303566583367 us; error 33 us; skew 500 ppm
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303566, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:06.586715 31101 init.cc:377] Logged in from keytab as oryx/127.21.81.193@KRBTEST.COM (short username oryx)
I20260809 19:26:06.587278 31101 webserver.cc:460] Webserver started at http://127.21.81.193:33551/ using document root <none> and password file <none>
I20260809 19:26:06.587538 31101 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:06.587625 31101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:06.589411 31101 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:26:06.590178 31136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.590368 31101 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:26:06.590473 31101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "17007b4f301d40c3bce58b9eaeb353d1"
format_stamp: "Formatted at 2026-08-09 19:26:05 on dist-test-slave-njxd"
server_key: "681c26d1c7151224b465afdeb2ce1dfe"
server_key_iv: "60ed5f54f68b0993e5070d8c576794bd"
server_key_version: "encryptionkey@0"
I20260809 19:26:06.590832 31101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:06.620285 31101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:06.622660 31101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:06.622816 31101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:06.623078 31101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:26:06.623394 31101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:06.623427 31101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.623450 31101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:06.623492 31101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.629237 31101 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:38589
I20260809 19:26:06.629341 31254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:38589 every 8 connection(s)
I20260809 19:26:06.629709 31101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:26:06.633942 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 31101
I20260809 19:26:06.634275 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:34261
--local_ip_for_outbound_sockets=127.21.81.194
--tserver_master_addrs=127.21.81.254:32983
--webserver_port=40815
--webserver_interface=127.21.81.194
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:33277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.81.194
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:06.638685 31257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:06.630450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:56887)
0809 19:26:06.630632 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:06.630702 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:06.631222 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:06.631653 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:06.631656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:06.632250 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:06.632435 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.632439 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.633533 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.633534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.634816 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.634820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.635409 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.635411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.635558 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.636079 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:06.636089 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:06.638506 (+ 2417us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
W20260809 19:26:06.638901 31258 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:06.630543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:56887 (local address 127.21.81.254:32983)
0809 19:26:06.630742 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:06.630751 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:06.630797 (+ 46us) server_negotiation.cc:407] Connection header received
0809 19:26:06.631423 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:06.631429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:06.631478 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:06.631611 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:06.632557 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.633420 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.634906 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.635315 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.635407 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.638692 (+ 3285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:56887: BlockingRecv error: recv got EOF from 127.21.81.193:56887 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
W20260809 19:26:06.639019 31255 heartbeater.cc:644] Failed to heartbeat to 127.21.81.254:32983 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.81.254:32983: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:06.643872 31257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:06.639325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:44045)
0809 19:26:06.639371 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:06.639382 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:06.639421 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:06.639690 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:06.639691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:06.639826 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:06.639870 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.639871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.640500 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.640501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.641016 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.641017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.641372 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.641373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.641438 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.641897 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:06.641903 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:06.643786 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 19:26:06.643929 31258 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:06.639400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:44045 (local address 127.21.81.254:32983)
0809 19:26:06.639583 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:06.639585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:06.639592 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:26:06.639611 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:06.639611 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:06.639620 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:06.639685 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:06.639942 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.640423 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.641098 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.641303 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.641375 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.643868 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:44045: BlockingRecv error: recv got EOF from 127.21.81.193:44045 (error 108)
Metrics: {"server-negotiator.queue_time_us":165}
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:06.648335 31257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:06.644140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:33465)
0809 19:26:06.644205 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:06.644218 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:06.644255 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:06.644438 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:06.644439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:06.644566 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:06.644594 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.644595 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.645214 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.645215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.645786 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.645788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.646120 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.646121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.646172 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.646576 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:06.646581 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:06.648251 (+ 1670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 19:26:06.648386 31258 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:06.644275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:33465 (local address 127.21.81.254:32983)
0809 19:26:06.644332 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:06.644333 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:06.644341 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:26:06.644366 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:06.644366 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:06.644375 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:06.644433 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:06.644666 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.645117 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.645864 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.646053 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.646123 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.648321 (+ 2198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:33465: BlockingRecv error: recv got EOF from 127.21.81.193:33465 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 19:26:06.648631 31255 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:26:06.719722 31259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:06.719941 31259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:06.719965 31259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:06.721143 31259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:06.721187 31259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:06.721235 31259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:26:06.722808 31259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:33277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.194.keytab
--principal=oryx/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:34261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=40815
--tserver_master_addrs=127.21.81.254:32983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:06.723043 31259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:06.723275 31259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:06.728459 31264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:06.728466 31267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:06.728483 31259 server_base.cc:1034] running on GCE node
W20260809 19:26:06.728518 31265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:06.728926 31259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:06.729200 31259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:06.730371 31259 hybrid_clock.cc:648] HybridClock initialized: now 1786303566730362 us; error 33 us; skew 500 ppm
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303566, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:06.732882 31259 init.cc:377] Logged in from keytab as oryx/127.21.81.194@KRBTEST.COM (short username oryx)
I20260809 19:26:06.733335 31259 webserver.cc:460] Webserver started at http://127.21.81.194:40815/ using document root <none> and password file <none>
I20260809 19:26:06.733522 31259 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:06.733631 31259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:06.734934 31259 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:26:06.735478 31274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.735648 31259 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:26:06.735741 31259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "64b21a6dbb6547d9a8b30e22e111b2d4"
format_stamp: "Formatted at 2026-08-09 19:26:06 on dist-test-slave-njxd"
server_key: "cd372e07780a170c71f7e567fc068c26"
server_key_iv: "1dc9043e9bfa282701be0f29b697395f"
server_key_version: "encryptionkey@0"
I20260809 19:26:06.736063 31259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:06.756454 31259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:06.758771 31259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:06.758885 31259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:06.759079 31259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:26:06.759397 31259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:06.759433 31259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.759454 31259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:06.759496 31259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.765653 31259 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:34261
I20260809 19:26:06.765699 31400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:34261 every 8 connection(s)
I20260809 19:26:06.766065 31259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:26:06.770706 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 31259
I20260809 19:26:06.771255 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:32855
--local_ip_for_outbound_sockets=127.21.81.195
--tserver_master_addrs=127.21.81.254:32983
--webserver_port=35117
--webserver_interface=127.21.81.195
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:33277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.81.195
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 19:26:06.773682 31258 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:06.766777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:57603 (local address 127.21.81.254:32983)
0809 19:26:06.766839 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:06.766841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:06.767056 (+ 215us) server_negotiation.cc:407] Connection header received
0809 19:26:06.767728 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:06.767730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:06.767745 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:06.767801 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:06.768713 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.769207 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.770230 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.770415 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.770493 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.773625 (+ 3132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:57603: BlockingRecv error: recv got EOF from 127.21.81.194:57603 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:26:06.773718 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:06.766749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:57603)
0809 19:26:06.766927 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:06.766969 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:06.767599 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:06.767884 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:06.767888 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:06.768424 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:06.768606 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.768610 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.769289 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.769290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.770151 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.770154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.770511 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.770512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.770611 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.771073 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:06.771082 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:06.773497 (+ 2415us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":104,"threads_started":1}
W20260809 19:26:06.774415 31401 heartbeater.cc:644] Failed to heartbeat to 127.21.81.254:32983 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.81.254:32983: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:06.778927 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:06.774682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:38683)
0809 19:26:06.774727 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:06.774737 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:06.774774 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:06.774903 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:06.774903 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:06.775038 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:06.775082 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.775084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.775703 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.775704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.776216 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.776217 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.776555 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.776556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.776622 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.777075 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:06.777080 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:06.778840 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 19:26:06.778985 31258 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:06.774736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:38683 (local address 127.21.81.254:32983)
0809 19:26:06.774778 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:06.774780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:06.774798 (+ 18us) server_negotiation.cc:407] Connection header received
0809 19:26:06.774824 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:06.774825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:06.774833 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:06.774900 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:06.775165 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.775631 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.776297 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.776487 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.776559 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.778919 (+ 2360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:38683: BlockingRecv error: recv got EOF from 127.21.81.194:38683 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:06.783351 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:06.779283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:50301)
0809 19:26:06.779326 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:06.779336 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:06.779374 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:06.779494 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:06.779495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:06.779617 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:06.779644 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.779645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.780252 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.780253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.780809 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.780811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.781151 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.781152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.781211 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.781677 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:06.781681 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:06.783281 (+ 1600us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 19:26:06.783433 31258 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:06.779325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:50301 (local address 127.21.81.254:32983)
0809 19:26:06.779361 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:06.779363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:06.779371 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:26:06.779426 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:06.779427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:06.779435 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:06.779490 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:06.779719 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.780175 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.780886 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.781082 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.781153 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.783359 (+ 2206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:50301: BlockingRecv error: recv got EOF from 127.21.81.194:50301 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 19:26:06.783511 31401 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:26:06.856137 31405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:06.856343 31405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:06.856366 31405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:06.857659 31405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:06.857702 31405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:06.857748 31405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:26:06.859205 31405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:33277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/krb5kdc/oryx_127.21.81.195.keytab
--principal=oryx/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:32855
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=35117
--tserver_master_addrs=127.21.81.254:32983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:06.859454 31405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:06.859679 31405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:06.864714 31410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:06.864709 31411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:06.864710 31413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:06.864914 31405 server_base.cc:1034] running on GCE node
I20260809 19:26:06.865144 31405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:06.865372 31405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:06.866518 31405 hybrid_clock.cc:648] HybridClock initialized: now 1786303566866488 us; error 44 us; skew 500 ppm
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303566, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:06.868901 31405 init.cc:377] Logged in from keytab as oryx/127.21.81.195@KRBTEST.COM (short username oryx)
I20260809 19:26:06.869292 31405 webserver.cc:460] Webserver started at http://127.21.81.195:35117/ using document root <none> and password file <none>
I20260809 19:26:06.869470 31405 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:06.869524 31405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:06.870839 31405 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:26:06.871366 31421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.871515 31405 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:26:06.871596 31405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "33fb0499865840469df2081c305215c5"
format_stamp: "Formatted at 2026-08-09 19:26:06 on dist-test-slave-njxd"
server_key: "cfbfa9792d8cefb264b8817aeff94c59"
server_key_iv: "b900046e175a023dc3ac41cad487e4e1"
server_key_version: "encryptionkey@0"
I20260809 19:26:06.871901 31405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:06.887468 31405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:06.889777 31405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:06.889892 31405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:06.890093 31405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:26:06.890343 31405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:06.890374 31405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.890396 31405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:06.890408 31405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:06.896433 31405 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:32855
I20260809 19:26:06.896509 31540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:32855 every 8 connection(s)
I20260809 19:26:06.896879 31405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:26:06.897333 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 31405
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 19:26:06.905726 31258 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:06.897754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:41565 (local address 127.21.81.254:32983)
0809 19:26:06.897822 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:06.897825 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:06.898133 (+ 308us) server_negotiation.cc:407] Connection header received
0809 19:26:06.898733 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:06.898734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:06.898743 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:06.898797 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:06.900035 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.900668 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.901935 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.902102 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.902174 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.905660 (+ 3486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:41565: BlockingRecv error: recv got EOF from 127.21.81.195:41565 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:26:06.905747 31543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:06.897757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:41565)
0809 19:26:06.897973 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:06.898023 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:06.898648 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:06.898910 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:06.898914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:06.899714 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:06.899954 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.899960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.900750 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.900751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.901842 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.901844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.902212 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.902213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.902316 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.902781 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:06.902792 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:06.905524 (+ 2732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":88,"threads_started":1}
W20260809 19:26:06.905993 31541 heartbeater.cc:644] Failed to heartbeat to 127.21.81.254:32983 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.81.254:32983: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
I20260809 19:26:06.907272 31545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:06.897904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.21.81.254:32983)
0809 19:26:06.898003 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:06.898006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:06.898899 (+ 893us) server_negotiation.cc:407] Connection header received
0809 19:26:06.898968 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:06.898969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:06.898975 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:06.899010 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:06.899813 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.900524 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.901634 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.901825 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.901872 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.903424 (+ 1552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:06.903428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:06.903428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:06.903437 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:06.905441 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:06.906135 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:06.906138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:06.906138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:06.906269 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:06.906603 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:06.906604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:06.906605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:06.906869 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:06.906966 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:06.907125 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 19:26:06.907208 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":55,"threads_started":1}
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:06.910392 31543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:06.906227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:45749)
0809 19:26:06.906272 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:06.906282 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:06.906315 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:06.906449 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:06.906449 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:06.906571 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:06.906626 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.906627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.907430 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.907431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.908077 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.908080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.908402 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.908403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.908449 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.908854 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:06.908859 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:06.910350 (+ 1491us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 19:26:06.910485 31258 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:06.906286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:45749 (local address 127.21.81.254:32983)
0809 19:26:06.906335 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:06.906337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:06.906361 (+ 24us) server_negotiation.cc:407] Connection header received
0809 19:26:06.906385 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:06.906385 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:06.906393 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:06.906434 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:06.906698 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.907313 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.908170 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.908347 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.908394 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.910425 (+ 2031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:45749: BlockingRecv error: recv got EOF from 127.21.81.195:45749 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:06 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:06.914458 31543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:06.910672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:39905)
0809 19:26:06.910731 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:06.910742 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:06.910787 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:06.910908 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:06.910909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:06.911032 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:06.911054 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.911055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.911631 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.911632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.912105 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:06.912107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.912444 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.912445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:06.912495 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.912815 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:06.912819 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:06.914384 (+ 1565us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 19:26:06.914500 31258 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:06.910728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:39905 (local address 127.21.81.254:32983)
0809 19:26:06.910762 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:06.910763 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:06.910774 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:26:06.910843 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:06.910844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:06.910853 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:06.910900 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:06.911130 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.911552 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.912190 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:06.912365 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:06.912433 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:06.914445 (+ 2012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:39905: BlockingRecv error: recv got EOF from 127.21.81.195:39905 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 19:26:06.914633 31541 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 19:26:07 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:07 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:07.653760 31546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:07.649048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:33891)
0809 19:26:07.649214 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:07.649226 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:07.649285 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:07.649431 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:07.649432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:07.649670 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:07.649727 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:07.649729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:07.650482 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:07.650483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:07.651037 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:07.651038 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:07.651365 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:07.651365 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:07.651405 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:07.651841 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:07.651845 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:07.653688 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":100,"threads_started":1}
W20260809 19:26:07.653865 31547 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:07.649142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:33891 (local address 127.21.81.254:32983)
0809 19:26:07.649286 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:07.649288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:07.649327 (+ 39us) server_negotiation.cc:407] Connection header received
0809 19:26:07.649351 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:07.649352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:07.649362 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:07.649414 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:07.649798 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:07.650406 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:07.651109 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:07.651302 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:07.651352 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:07.653788 (+ 2436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:33891: BlockingRecv error: recv got EOF from 127.21.81.193:33891 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
Aug 09 19:26:07 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:07 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:07.788749 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:07.784023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:42353)
0809 19:26:07.784174 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:07.784184 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:07.784242 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:07.784364 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:07.784365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:07.784539 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:07.784591 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:07.784593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:07.785468 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:07.785469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:07.786088 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:07.786090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:07.786428 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:07.786429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:07.786468 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:07.786944 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:07.786948 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:07.788675 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
W20260809 19:26:07.788883 31547 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:07.784109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:42353 (local address 127.21.81.254:32983)
0809 19:26:07.784171 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:07.784174 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:07.784240 (+ 66us) server_negotiation.cc:407] Connection header received
0809 19:26:07.784283 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:07.784283 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:07.784293 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:07.784348 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:07.784694 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:07.785401 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:07.786171 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:07.786366 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:07.786415 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:07.788798 (+ 2383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:42353: BlockingRecv error: recv got EOF from 127.21.81.194:42353 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 09 19:26:07 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:07 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:07.919303 31549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:07.915000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:36521)
0809 19:26:07.915163 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:07.915173 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:07.915232 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:07.915360 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:07.915360 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:07.915527 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:07.915576 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:07.915577 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:07.916199 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:07.916200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:07.916728 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:07.916729 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:07.917037 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:07.917038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:07.917076 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:07.917517 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:07.917521 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:07.919237 (+ 1716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
W20260809 19:26:07.919400 31547 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:07.915076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:36521 (local address 127.21.81.254:32983)
0809 19:26:07.915135 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:07.915137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:07.915233 (+ 96us) server_negotiation.cc:407] Connection header received
0809 19:26:07.915288 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:07.915289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:07.915301 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:07.915341 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:07.915645 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:07.916128 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:07.916797 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:07.916977 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:07.917024 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:07.919334 (+ 2310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:36521: BlockingRecv error: recv got EOF from 127.21.81.195:36521 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 19:26:08 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:08 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:08.659035 31550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:08.654320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:56907)
0809 19:26:08.654457 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:08.654468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:08.654519 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:08.654679 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:08.654680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:08.654853 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:08.654906 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:08.654907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:08.655592 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:08.655594 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:08.656141 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:08.656144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:08.656484 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:08.656486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:08.656534 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:08.657028 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:08.657033 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:08.658930 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
W20260809 19:26:08.659157 31551 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:08.654391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:56907 (local address 127.21.81.254:32983)
0809 19:26:08.654525 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:08.654527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:08.654538 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:26:08.654574 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:08.654574 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:08.654586 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:08.654657 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:08.654976 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:08.655489 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:08.656222 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:08.656410 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:08.656461 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:08.659067 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:56907: BlockingRecv error: recv got EOF from 127.21.81.193:56907 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
Aug 09 19:26:08 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:08 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:08.794188 31552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:08.789353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:45755)
0809 19:26:08.789492 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:08.789504 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:08.789607 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:08.789773 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:08.789774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:08.789958 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:08.790007 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:08.790009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:08.790706 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:08.790708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:08.791233 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:08.791234 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:08.791603 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:08.791604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:08.791673 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:08.792132 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:08.792136 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:08.794089 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
W20260809 19:26:08.794247 31551 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:08.789417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:45755 (local address 127.21.81.254:32983)
0809 19:26:08.789488 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:08.789490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:08.789566 (+ 76us) server_negotiation.cc:407] Connection header received
0809 19:26:08.789683 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:08.789683 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:08.789695 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:08.789766 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:08.790095 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:08.790608 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:08.791317 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:08.791515 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:08.791600 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:08.794179 (+ 2579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:45755: BlockingRecv error: recv got EOF from 127.21.81.194:45755 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 19:26:08 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:08 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:08.924317 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:08.919844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:33465)
0809 19:26:08.919984 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:08.919997 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:08.920064 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:08.920192 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:08.920200 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:08.920385 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:08.920435 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:08.920437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:08.921093 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:08.921095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:08.921648 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:08.921649 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:08.922012 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:08.922013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:08.922073 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:08.922496 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:08.922501 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:08.924223 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
W20260809 19:26:08.924358 31551 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:08.919957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:33465 (local address 127.21.81.254:32983)
0809 19:26:08.920006 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:08.920008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:08.920046 (+ 38us) server_negotiation.cc:407] Connection header received
0809 19:26:08.920116 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:08.920117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:08.920131 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:08.920175 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:08.920516 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:08.921011 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:08.921737 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:08.921940 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:08.922013 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:08.924298 (+ 2285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:33465: BlockingRecv error: recv got EOF from 127.21.81.195:33465 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 19:26:09 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:09 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:09.664381 31554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:09.659674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:51667)
0809 19:26:09.659847 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:09.659858 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:09.659914 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:09.660075 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:09.660076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:09.660255 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:09.660306 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:09.660308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:09.660943 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:09.660943 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:09.661465 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:09.661467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:09.661821 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:09.661822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:09.661871 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:09.662374 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:09.662379 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:09.664287 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":106,"threads_started":1}
W20260809 19:26:09.664472 31555 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:09.659805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:51667 (local address 127.21.81.254:32983)
0809 19:26:09.659941 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:09.659943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:09.659953 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:26:09.659983 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:09.659984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:09.659994 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:09.660052 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:09.660375 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:09.660876 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:09.661537 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:09.661748 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:09.661804 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:09.664387 (+ 2583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:51667: BlockingRecv error: recv got EOF from 127.21.81.193:51667 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
Aug 09 19:26:09 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:09 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:09.799172 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:09.794705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:36415)
0809 19:26:09.794867 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:09.794878 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:09.794937 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:09.795064 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:09.795065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:09.795244 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:09.795293 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:09.795295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:09.795928 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:09.795929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:09.796441 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:09.796443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:09.796785 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:09.796786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:09.796827 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:09.797260 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:09.797265 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:09.799114 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
W20260809 19:26:09.799338 31555 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:09.794752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:36415 (local address 127.21.81.254:32983)
0809 19:26:09.794817 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:09.794819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:09.794933 (+ 114us) server_negotiation.cc:407] Connection header received
0809 19:26:09.794985 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:09.794986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:09.794996 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:09.795045 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:09.795350 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:09.795862 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:09.796512 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:09.796719 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:09.796774 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:09.799272 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:36415: BlockingRecv error: recv got EOF from 127.21.81.194:36415 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 19:26:09 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:09 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:09.929895 31557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:09.924809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:49889)
0809 19:26:09.924981 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:09.925006 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:09.925088 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:09.925250 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:09.925251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:09.925443 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:09.925493 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:09.925495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:09.926223 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:09.926224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:09.926756 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:09.926757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:09.927138 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:09.927139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:09.927207 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:09.927636 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:09.927641 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:09.929837 (+ 2196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":112,"threads_started":1}
W20260809 19:26:09.930029 31555 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:09.924975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:49889 (local address 127.21.81.254:32983)
0809 19:26:09.925041 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:09.925043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:09.925052 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:26:09.925171 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:09.925171 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:09.925185 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:09.925231 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:09.925595 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:09.926122 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:09.926840 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:09.927046 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:09.927131 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:09.929943 (+ 2812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:49889: BlockingRecv error: recv got EOF from 127.21.81.195:49889 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 19:26:10 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:10 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:10.675788 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:10.664940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:48243)
0809 19:26:10.665125 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:10.665141 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:10.665211 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:10.665376 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:10.665377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:10.665550 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:10.665655 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:10.665657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:10.666345 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:10.666347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:10.666886 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:10.666887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:10.667239 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:10.667240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:10.667285 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:10.673714 (+ 6429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:10.673721 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:10.675687 (+ 1966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":115,"threads_started":1}
W20260809 19:26:10.675901 31559 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:10.665022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:48243 (local address 127.21.81.254:32983)
0809 19:26:10.665162 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:10.665164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:10.665192 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:26:10.665274 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:10.665274 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:10.665285 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:10.665354 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:10.665725 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:10.666241 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:10.666968 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:10.667167 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:10.667222 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:10.675782 (+ 8560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:48243: BlockingRecv error: recv got EOF from 127.21.81.193:48243 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
Aug 09 19:26:10 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:10 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:10.804149 31560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:10.799620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:59537)
0809 19:26:10.799787 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:10.799798 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:10.799865 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:10.799990 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:10.799991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:10.800159 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:10.800217 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:10.800219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:10.800830 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:10.800831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:10.801360 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:10.801361 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:10.801717 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:10.801718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:10.801760 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:10.802264 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:10.802272 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:10.804089 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":111,"threads_started":1}
W20260809 19:26:10.804260 31559 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:10.799677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:59537 (local address 127.21.81.254:32983)
0809 19:26:10.799736 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:10.799738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:10.799860 (+ 122us) server_negotiation.cc:407] Connection header received
0809 19:26:10.799908 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:10.799909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:10.799920 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:10.799973 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:10.800273 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:10.800764 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:10.801424 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:10.801654 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:10.801704 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:10.804187 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:59537: BlockingRecv error: recv got EOF from 127.21.81.194:59537 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 19:26:10 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:10 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:10.935181 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:10.930417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:44663)
0809 19:26:10.930586 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:10.930604 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:10.930722 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:10.930861 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:10.930862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:10.931045 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:10.931102 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:10.931104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:10.931760 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:10.931761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:10.932287 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:10.932289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:10.932656 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:10.932657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:10.932718 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:10.933204 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:10.933208 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:10.935109 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":72,"threads_started":1}
W20260809 19:26:10.935285 31559 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:10.930485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:44663 (local address 127.21.81.254:32983)
0809 19:26:10.930566 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:10.930568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:10.930678 (+ 110us) server_negotiation.cc:407] Connection header received
0809 19:26:10.930786 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:10.930787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:10.930802 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:10.930855 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:10.931178 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:10.931671 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:10.932367 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:10.932572 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:10.932654 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:10.935201 (+ 2547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:44663: BlockingRecv error: recv got EOF from 127.21.81.195:44663 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 19:26:11 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:11 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:11.684937 31563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:11.676459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:35667)
0809 19:26:11.676697 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:11.676708 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:11.676799 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:11.677030 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:11.677031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:11.677525 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:11.677649 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:11.677654 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:11.678931 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:11.678932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:11.679852 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:11.679854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:11.680487 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:11.680488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:11.680549 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:11.681371 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:11.681388 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:11.684794 (+ 3406us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":158,"threads_started":1}
W20260809 19:26:11.685213 31564 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:11.676573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:35667 (local address 127.21.81.254:32983)
0809 19:26:11.676821 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:11.676823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:11.676839 (+ 16us) server_negotiation.cc:407] Connection header received
0809 19:26:11.676884 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:11.676885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:11.676905 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:11.677020 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:11.677812 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:11.678782 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:11.680010 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:11.680398 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:11.680472 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:11.685038 (+ 4566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:35667: BlockingRecv error: recv got EOF from 127.21.81.193:35667 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":147,"threads_started":1}
Aug 09 19:26:11 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:11 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:11.813763 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:11.804779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:52741)
0809 19:26:11.804987 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:11.805006 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:11.805121 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:11.805360 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:11.805360 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:11.805893 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:11.806031 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:11.806041 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:11.807893 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:11.807894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:11.808917 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:11.808920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:11.809423 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:11.809424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:11.809475 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:11.810223 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:11.810235 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:11.813644 (+ 3409us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":110,"threads_started":1}
W20260809 19:26:11.813910 31564 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:11.804945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:52741 (local address 127.21.81.254:32983)
0809 19:26:11.805034 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:11.805045 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:11.805086 (+ 41us) server_negotiation.cc:407] Connection header received
0809 19:26:11.805221 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:11.805222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:11.805242 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:11.805323 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:11.806193 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:11.807750 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:11.809042 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:11.809333 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:11.809389 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:11.813814 (+ 4425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:52741: BlockingRecv error: recv got EOF from 127.21.81.194:52741 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 19:26:11 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:11 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:11.944497 31566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:11.935891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:51633)
0809 19:26:11.936151 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:11.936163 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:11.936281 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:11.936667 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:11.936668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:11.937104 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:11.937291 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:11.937294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:11.938516 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:11.938517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:11.939793 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:11.939796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:11.940300 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:11.940301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:11.940358 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:11.941159 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:11.941164 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:11.944420 (+ 3256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":156,"threads_started":1}
W20260809 19:26:11.944654 31564 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:11.935987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:51633 (local address 127.21.81.254:32983)
0809 19:26:11.936127 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:11.936138 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:11.936274 (+ 136us) server_negotiation.cc:407] Connection header received
0809 19:26:11.936418 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:11.936419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:11.936441 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:11.936635 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:11.937407 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:11.938436 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:11.939905 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:11.940207 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:11.940299 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:11.944542 (+ 4243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:51633: BlockingRecv error: recv got EOF from 127.21.81.195:51633 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 19:26:12 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:12 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:12.693984 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:12.685840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:55065)
0809 19:26:12.686063 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:12.686078 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:12.686180 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:12.686434 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:12.686442 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:12.686851 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:12.687046 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:12.687058 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:12.688304 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:12.688306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:12.689298 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:12.689300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:12.689848 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:12.689849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:12.689921 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:12.690766 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:12.690785 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:12.693896 (+ 3111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":137,"threads_started":1}
W20260809 19:26:12.694324 31568 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:12.685947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:55065 (local address 127.21.81.254:32983)
0809 19:26:12.686201 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:12.686202 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:12.686224 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:26:12.686292 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:12.686293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:12.686313 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:12.686429 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:12.687197 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:12.688202 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:12.689406 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:12.689739 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:12.689848 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:12.694109 (+ 4261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:55065: BlockingRecv error: recv got EOF from 127.21.81.193:55065 (error 108)
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":146,"threads_started":1}
Aug 09 19:26:12 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:12 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:12.819243 31569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:12.814441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:55269)
0809 19:26:12.814640 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:12.814652 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:12.814755 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:12.814927 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:12.814928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:12.815124 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:12.815177 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:12.815179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:12.815860 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:12.815862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:12.816402 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:12.816403 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:12.816766 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:12.816767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:12.816827 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:12.817331 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:12.817339 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:12.819146 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":125,"threads_started":1}
W20260809 19:26:12.819298 31568 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:12.814531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:55269 (local address 127.21.81.254:32983)
0809 19:26:12.814638 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:12.814640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:12.814735 (+ 95us) server_negotiation.cc:407] Connection header received
0809 19:26:12.814834 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:12.814835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:12.814854 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:12.814920 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:12.815263 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:12.815768 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:12.816489 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:12.816688 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:12.816767 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:12.819236 (+ 2469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:55269: BlockingRecv error: recv got EOF from 127.21.81.194:55269 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 19:26:12 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:12 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:12.949477 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:12.945152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:37023)
0809 19:26:12.945326 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:12.945345 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:12.945403 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:12.945520 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:12.945520 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:12.945716 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:12.945769 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:12.945770 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:12.946388 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:12.946389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:12.946915 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:12.946916 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:12.947218 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:12.947219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:12.947259 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:12.947689 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:12.947693 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:12.949417 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":116,"threads_started":1}
W20260809 19:26:12.949558 31568 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:12.945240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:37023 (local address 127.21.81.254:32983)
0809 19:26:12.945308 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:12.945310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:12.945405 (+ 95us) server_negotiation.cc:407] Connection header received
0809 19:26:12.945445 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:12.945446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:12.945459 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:12.945514 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:12.945833 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:12.946318 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:12.946984 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:12.947156 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:12.947206 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:12.949509 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:37023: BlockingRecv error: recv got EOF from 127.21.81.195:37023 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 19:26:13 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:13 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:13.699901 31571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:13.694488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:55169)
0809 19:26:13.694656 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:13.694675 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:13.694786 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:13.694994 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:13.694994 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:13.695167 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:13.695222 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:13.695223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:13.695888 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:13.695889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:13.696424 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:13.696425 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:13.696773 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:13.696774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:13.696815 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:13.697382 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:13.697387 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:13.699791 (+ 2404us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":98,"threads_started":1}
W20260809 19:26:13.699946 31572 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:13.694707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:55169 (local address 127.21.81.254:32983)
0809 19:26:13.694861 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:13.694863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:13.694876 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:26:13.694900 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:13.694900 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:13.694910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:13.694975 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:13.695292 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:13.695820 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:13.696499 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:13.696705 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:13.696760 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:13.699871 (+ 3111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:55169: BlockingRecv error: recv got EOF from 127.21.81.193:55169 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":101,"threads_started":1}
Aug 09 19:26:13 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:13 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:13.824918 31573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:13.819773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:40865)
0809 19:26:13.819975 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:13.819989 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:13.820083 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:13.820252 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:13.820253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:13.820448 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:13.820499 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:13.820501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:13.821271 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:13.821273 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:13.821891 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:13.821893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:13.822286 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:13.822287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:13.822345 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:13.822945 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:13.822950 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:13.824808 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":97,"threads_started":1}
W20260809 19:26:13.824971 31572 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:13.819870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:40865 (local address 127.21.81.254:32983)
0809 19:26:13.819980 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:13.819982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:13.820064 (+ 82us) server_negotiation.cc:407] Connection header received
0809 19:26:13.820169 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:13.820169 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:13.820184 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:13.820234 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:13.820587 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:13.821145 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:13.821983 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:13.822205 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:13.822287 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:13.824905 (+ 2618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:40865: BlockingRecv error: recv got EOF from 127.21.81.194:40865 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 19:26:13 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:13 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:13.954589 31574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:13.950009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:38483)
0809 19:26:13.950170 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:13.950180 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:13.950264 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:13.950388 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:13.950389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:13.950553 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:13.950606 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:13.950608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:13.951244 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:13.951244 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:13.951756 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:13.951757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:13.952050 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:13.952051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:13.952089 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:13.952596 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:13.952601 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:13.954481 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":104,"threads_started":1}
W20260809 19:26:13.954741 31572 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:13.950056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:38483 (local address 127.21.81.254:32983)
0809 19:26:13.950104 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:13.950106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:13.950266 (+ 160us) server_negotiation.cc:407] Connection header received
0809 19:26:13.950307 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:13.950308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:13.950320 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:13.950375 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:13.950679 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:13.951179 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:13.951820 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:13.951993 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:13.952040 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:13.954660 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:38483: BlockingRecv error: recv got EOF from 127.21.81.195:38483 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 19:26:14 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:14 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:14.704979 31575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:14.700387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:59949)
0809 19:26:14.700530 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:14.700541 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:14.700616 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:14.700789 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:14.700789 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:14.700964 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:14.701015 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:14.701016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:14.701708 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:14.701709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:14.702224 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:14.702226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:14.702565 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:14.702565 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:14.702604 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:14.703067 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:14.703072 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:14.704910 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":56,"threads_started":1}
W20260809 19:26:14.705152 31576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:14.700529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:59949 (local address 127.21.81.254:32983)
0809 19:26:14.700665 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:14.700667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:14.700674 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:26:14.700701 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:14.700701 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:14.700712 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:14.700769 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:14.701078 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:14.701563 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:14.702311 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:14.702504 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:14.702551 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:14.705063 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:59949: BlockingRecv error: recv got EOF from 127.21.81.193:59949 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
Aug 09 19:26:14 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:14 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:14.830405 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:14.825404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:34501)
0809 19:26:14.825561 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:14.825592 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:14.825651 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:14.825780 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:14.825781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:14.825954 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:14.826007 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:14.826009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:14.826769 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:14.826770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:14.827293 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:14.827295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:14.827610 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:14.827611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:14.827654 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:14.828092 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:14.828096 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:14.830342 (+ 2246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
W20260809 19:26:14.830528 31576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:14.825467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:34501 (local address 127.21.81.254:32983)
0809 19:26:14.825554 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:14.825555 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:14.825649 (+ 94us) server_negotiation.cc:407] Connection header received
0809 19:26:14.825698 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:14.825699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:14.825709 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:14.825769 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:14.826070 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:14.826697 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:14.827367 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:14.827546 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:14.827594 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:14.830448 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:34501: BlockingRecv error: recv got EOF from 127.21.81.194:34501 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 19:26:14 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:14 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:14.959697 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:14.955229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:58929)
0809 19:26:14.955377 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:14.955388 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:14.955459 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:14.955603 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:14.955604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:14.955768 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:14.955822 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:14.955823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:14.956450 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:14.956454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:14.956966 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:14.956967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:14.957295 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:14.957296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:14.957335 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:14.957889 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:14.957894 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:14.959638 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":71,"threads_started":1}
W20260809 19:26:14.959904 31576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:14.955276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:58929 (local address 127.21.81.254:32983)
0809 19:26:14.955332 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:14.955334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:14.955440 (+ 106us) server_negotiation.cc:407] Connection header received
0809 19:26:14.955515 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:14.955515 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:14.955531 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:14.955583 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:14.955884 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:14.956383 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:14.957042 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:14.957228 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:14.957282 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:14.959816 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:58929: BlockingRecv error: recv got EOF from 127.21.81.195:58929 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 09 19:26:15 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:15 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:15.713063 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:15.705539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:49307)
0809 19:26:15.705704 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:15.705718 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:15.705770 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:15.705929 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:15.705930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:15.706112 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:15.706165 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:15.706167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:15.706818 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:15.706819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:15.707349 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:15.707350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:15.707678 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:15.707679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:15.707723 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:15.710915 (+ 3192us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:15.710920 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:15.712987 (+ 2067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
W20260809 19:26:15.713145 31580 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:15.705658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:49307 (local address 127.21.81.254:32983)
0809 19:26:15.705797 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:15.705799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:15.705808 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:26:15.705830 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:15.705831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:15.705843 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:15.705907 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:15.706236 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:15.706747 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:15.707424 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:15.707609 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:15.707660 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:15.713090 (+ 5430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:49307: BlockingRecv error: recv got EOF from 127.21.81.193:49307 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
Aug 09 19:26:15 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:15 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:15.835294 31581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:15.830891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:53583)
0809 19:26:15.831013 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:15.831030 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:15.831089 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:15.831197 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:15.831198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:15.831369 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:15.831438 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:15.831440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:15.832086 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:15.832087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:15.832598 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:15.832599 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:15.832916 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:15.832917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:15.832957 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:15.833426 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:15.833431 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:15.835226 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
W20260809 19:26:15.835392 31580 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:15.830970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:53583 (local address 127.21.81.254:32983)
0809 19:26:15.831019 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:15.831021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:15.831076 (+ 55us) server_negotiation.cc:407] Connection header received
0809 19:26:15.831139 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:15.831140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:15.831151 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:15.831186 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:15.831507 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:15.832019 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:15.832667 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:15.832852 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:15.832901 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:15.835323 (+ 2422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:53583: BlockingRecv error: recv got EOF from 127.21.81.194:53583 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 19:26:15 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:15 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:15.964784 31582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:15.960125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:44635)
0809 19:26:15.960295 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:15.960306 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:15.960369 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:15.960484 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:15.960485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:15.960681 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:15.960732 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:15.960733 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:15.961346 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:15.961346 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:15.961931 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:15.961933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:15.962259 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:15.962260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:15.962298 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:15.962784 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:15.962789 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:15.964698 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
W20260809 19:26:15.964908 31580 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:15.960221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:44635 (local address 127.21.81.254:32983)
0809 19:26:15.960278 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:15.960280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:15.960366 (+ 86us) server_negotiation.cc:407] Connection header received
0809 19:26:15.960412 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:15.960413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:15.960425 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:15.960472 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:15.960787 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:15.961280 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:15.962015 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:15.962190 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:15.962248 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:15.964829 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:44635: BlockingRecv error: recv got EOF from 127.21.81.195:44635 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 19:26:16 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:16 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:16.718492 31583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:16.713840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:59329)
0809 19:26:16.713997 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:16.714010 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:16.714069 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:16.714215 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:16.714216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:16.714400 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:16.714451 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:16.714452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:16.715107 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:16.715108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:16.715625 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:16.715626 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:16.715947 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:16.715948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:16.715987 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:16.716467 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:16.716472 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:16.718380 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
W20260809 19:26:16.718637 31584 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:16.713944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:59329 (local address 127.21.81.254:32983)
0809 19:26:16.714069 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:16.714071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:16.714115 (+ 44us) server_negotiation.cc:407] Connection header received
0809 19:26:16.714139 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:16.714139 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:16.714150 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:16.714197 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:16.714514 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:16.715035 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:16.715696 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:16.715883 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:16.715934 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:16.718492 (+ 2558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:59329: BlockingRecv error: recv got EOF from 127.21.81.193:59329 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
Aug 09 19:26:16 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:16 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:16.841054 31585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:16.835867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:57609)
0809 19:26:16.836036 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:16.836050 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:16.836156 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:16.836325 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:16.836326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:16.836513 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:16.836562 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:16.836564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:16.837308 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:16.837310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:16.838152 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:16.838155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:16.838525 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:16.838526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:16.838581 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:16.839102 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:16.839108 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:16.840956 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
W20260809 19:26:16.841100 31584 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:16.835960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:57609 (local address 127.21.81.254:32983)
0809 19:26:16.836040 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:16.836042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:16.836136 (+ 94us) server_negotiation.cc:407] Connection header received
0809 19:26:16.836247 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:16.836248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:16.836260 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:16.836312 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:16.836681 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:16.837221 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:16.838244 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:16.838452 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:16.838526 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:16.841037 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:57609: BlockingRecv error: recv got EOF from 127.21.81.194:57609 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 19:26:16 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:16 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:16.970407 31586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:16.965446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:47177)
0809 19:26:16.965616 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:16.965627 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:16.965680 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:16.965831 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:16.965832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:16.966003 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:16.966056 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:16.966057 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:16.966796 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:16.966797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:16.967345 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:16.967347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:16.967802 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:16.967804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:16.967870 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:16.968391 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:16.968395 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:16.970345 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
W20260809 19:26:16.970479 31584 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:16.965626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:47177 (local address 127.21.81.254:32983)
0809 19:26:16.965715 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:16.965717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:16.965727 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:26:16.965750 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:16.965751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:16.965762 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:16.965816 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:16.966125 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:16.966720 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:16.967406 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:16.967694 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:16.967789 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:16.970433 (+ 2644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:47177: BlockingRecv error: recv got EOF from 127.21.81.195:47177 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 19:26:17 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:17 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:17.723453 31587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:17.718960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:60543)
0809 19:26:17.719081 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:17.719095 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:17.719188 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:17.719347 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:17.719347 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:17.719542 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:17.719593 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:17.719595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:17.720222 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:17.720223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:17.720769 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:17.720771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:17.721090 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:17.721091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:17.721135 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:17.721567 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:17.721593 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:17.723384 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
W20260809 19:26:17.723600 31588 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:17.719047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:60543 (local address 127.21.81.254:32983)
0809 19:26:17.719206 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:17.719208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:17.719239 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:26:17.719260 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:17.719260 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:17.719271 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:17.719321 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:17.719659 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:17.720157 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:17.720840 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:17.721029 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:17.721077 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:17.723504 (+ 2427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:60543: BlockingRecv error: recv got EOF from 127.21.81.193:60543 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
Aug 09 19:26:17 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:17 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:17.846246 31589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:17.841500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:54101)
0809 19:26:17.841742 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:17.841752 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:17.841834 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:17.841963 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:17.841963 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:17.842139 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:17.842184 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:17.842186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:17.842863 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:17.842863 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:17.843411 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:17.843413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:17.843739 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:17.843740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:17.843780 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:17.844209 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:17.844213 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:17.846133 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":122,"threads_started":1}
W20260809 19:26:17.846275 31588 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:17.841642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:54101 (local address 127.21.81.254:32983)
0809 19:26:17.841694 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:17.841696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:17.841834 (+ 138us) server_negotiation.cc:407] Connection header received
0809 19:26:17.841895 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:17.841896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:17.841908 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:17.841950 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:17.842251 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:17.842749 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:17.843480 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:17.843675 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:17.843726 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:17.846211 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:54101: BlockingRecv error: recv got EOF from 127.21.81.194:54101 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 19:26:17 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:17 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:17.975970 31590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:17.970978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:39675)
0809 19:26:17.971173 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:17.971189 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:17.971258 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:17.971386 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:17.971387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:17.971574 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:17.971631 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:17.971632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:17.972349 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:17.972350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:17.972900 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:17.972902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:17.973241 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:17.973242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:17.973292 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:17.973820 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:17.973826 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:17.975901 (+ 2075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":112,"threads_started":1}
W20260809 19:26:17.976066 31588 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:17.971045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:39675 (local address 127.21.81.254:32983)
0809 19:26:17.971117 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:17.971119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:17.971255 (+ 136us) server_negotiation.cc:407] Connection header received
0809 19:26:17.971304 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:17.971304 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:17.971316 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:17.971368 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:17.971699 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:17.972277 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:17.972972 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:17.973151 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:17.973212 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:17.975998 (+ 2786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:39675: BlockingRecv error: recv got EOF from 127.21.81.195:39675 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 19:26:18 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:18 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:18.728691 31591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:18.723990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:41115)
0809 19:26:18.724131 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:18.724150 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:18.724201 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:18.724397 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:18.724398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:18.724665 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:18.724717 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:18.724719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:18.725388 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:18.725389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:18.725962 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:18.725964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:18.726299 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:18.726300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:18.726340 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:18.726822 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:18.726827 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:18.728622 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
W20260809 19:26:18.728804 31592 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:18.724050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:41115 (local address 127.21.81.254:32983)
0809 19:26:18.724195 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:18.724197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:18.724208 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:26:18.724242 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:18.724243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:18.724256 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:18.724368 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:18.724794 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:18.725316 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:18.726036 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:18.726232 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:18.726285 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:18.728727 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:41115: BlockingRecv error: recv got EOF from 127.21.81.193:41115 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
Aug 09 19:26:18 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:18 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:18.851289 31593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:18.846726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:34077)
0809 19:26:18.846961 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:18.846973 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:18.847040 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:18.847167 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:18.847167 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:18.847346 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:18.847393 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:18.847395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:18.848019 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:18.848020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:18.848565 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:18.848566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:18.848912 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:18.848913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:18.848954 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:18.849401 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:18.849405 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:18.851219 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":183,"threads_started":1}
W20260809 19:26:18.851420 31592 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:18.846815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:34077 (local address 127.21.81.254:32983)
0809 19:26:18.846878 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:18.846881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:18.847043 (+ 162us) server_negotiation.cc:407] Connection header received
0809 19:26:18.847082 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:18.847083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:18.847094 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:18.847146 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:18.847457 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:18.847949 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:18.848650 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:18.848829 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:18.848898 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:18.851316 (+ 2418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:34077: BlockingRecv error: recv got EOF from 127.21.81.194:34077 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 19:26:18 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:18 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:18.980873 31594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:18.976423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:60759)
0809 19:26:18.976580 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:18.976597 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:18.976674 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:18.976804 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:18.976805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:18.976989 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:18.977041 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:18.977043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:18.977721 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:18.977722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:18.978253 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:18.978255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:18.978574 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:18.978575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:18.978628 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:18.979077 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:18.979081 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:18.980811 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
W20260809 19:26:18.980983 31592 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:18.976519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:60759 (local address 127.21.81.254:32983)
0809 19:26:18.976593 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:18.976595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:18.976673 (+ 78us) server_negotiation.cc:407] Connection header received
0809 19:26:18.976722 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:18.976722 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:18.976734 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:18.976780 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:18.977104 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:18.977634 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:18.978330 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:18.978510 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:18.978559 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:18.980909 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:60759: BlockingRecv error: recv got EOF from 127.21.81.195:60759 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 19:26:19 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:19 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:19.733824 31595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:19.729336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:47405)
0809 19:26:19.729486 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:19.729497 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:19.729557 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:19.729761 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:19.729762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:19.729946 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:19.729996 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:19.729998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:19.730627 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:19.730628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:19.731133 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:19.731134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:19.731472 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:19.731473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:19.731510 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:19.731994 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:19.732000 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:19.733765 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
W20260809 19:26:19.733978 31596 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:19.729382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:47405 (local address 127.21.81.254:32983)
0809 19:26:19.729527 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:19.729529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:19.729539 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:26:19.729660 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:19.729661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:19.729675 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:19.729736 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:19.730064 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:19.730557 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:19.731212 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:19.731405 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:19.731459 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:19.733888 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:47405: BlockingRecv error: recv got EOF from 127.21.81.193:47405 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
Aug 09 19:26:19 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:19 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:19.856652 31597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:19.851889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:45123)
0809 19:26:19.852087 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:19.852098 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:19.852167 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:19.852289 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:19.852290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:19.852479 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:19.852532 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:19.852534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:19.853318 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:19.853320 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:19.853993 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:19.853997 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:19.854398 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:19.854398 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:19.854445 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:19.854882 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:19.854887 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:19.856590 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":105,"threads_started":1}
W20260809 19:26:19.856752 31596 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:19.852034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:45123 (local address 127.21.81.254:32983)
0809 19:26:19.852109 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:19.852111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:19.852162 (+ 51us) server_negotiation.cc:407] Connection header received
0809 19:26:19.852221 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:19.852222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:19.852233 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:19.852277 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:19.852589 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:19.853220 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:19.854083 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:19.854323 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:19.854384 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:19.856683 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:45123: BlockingRecv error: recv got EOF from 127.21.81.194:45123 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 19:26:19 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:19 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:19.986603 31598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:19.981423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:60511)
0809 19:26:19.981654 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:19.981666 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:19.981750 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:19.981907 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:19.981908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:19.982111 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:19.982162 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:19.982164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:19.982830 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:19.982831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:19.983747 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:19.983751 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:19.984199 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:19.984200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:19.984278 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:19.984713 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:19.984718 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:19.986503 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":101,"threads_started":1}
W20260809 19:26:19.986656 31596 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:19.981502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:60511 (local address 127.21.81.254:32983)
0809 19:26:19.981597 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:19.981600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:19.981732 (+ 132us) server_negotiation.cc:407] Connection header received
0809 19:26:19.981812 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:19.981813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:19.981834 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:19.981897 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:19.982241 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:19.982737 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:19.983852 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:19.984136 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:19.984198 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:19.986594 (+ 2396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:60511: BlockingRecv error: recv got EOF from 127.21.81.195:60511 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 19:26:20 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:20 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:20.743799 31599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:20.734468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:41525)
0809 19:26:20.734675 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:20.734687 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:20.734745 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:20.734891 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:20.734892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:20.735071 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:20.735168 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:20.735170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:20.736065 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:20.736066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:20.736606 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:20.736607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:20.736959 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:20.736960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:20.737002 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:20.741872 (+ 4870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:20.741878 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:20.743736 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":131,"threads_started":1}
W20260809 19:26:20.743916 31600 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:20.734610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:41525 (local address 127.21.81.254:32983)
0809 19:26:20.734753 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:20.734755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:20.734766 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:26:20.734801 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:20.734801 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:20.734812 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:20.734874 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:20.735262 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:20.735990 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:20.736690 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:20.736889 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:20.736944 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:20.743841 (+ 6897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:41525: BlockingRecv error: recv got EOF from 127.21.81.193:41525 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
Aug 09 19:26:20 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:20 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:20.861717 31601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:20.857205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:49063)
0809 19:26:20.857340 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:20.857351 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:20.857417 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:20.857545 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:20.857545 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:20.857745 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:20.857793 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:20.857794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:20.858426 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:20.858427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:20.858945 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:20.858947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:20.859279 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:20.859279 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:20.859319 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:20.859755 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:20.859760 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:20.861643 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
W20260809 19:26:20.861778 31600 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:20.857277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:49063 (local address 127.21.81.254:32983)
0809 19:26:20.857327 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:20.857333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:20.857417 (+ 84us) server_negotiation.cc:407] Connection header received
0809 19:26:20.857465 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:20.857465 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:20.857476 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:20.857529 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:20.857858 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:20.858361 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:20.859020 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:20.859216 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:20.859268 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:20.861706 (+ 2438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:49063: BlockingRecv error: recv got EOF from 127.21.81.194:49063 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 19:26:20 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:20 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.195@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:20.991580 31602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:20.987133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.195:45137)
0809 19:26:20.987319 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:20.987330 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:20.987388 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:20.987525 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:20.987526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:20.987710 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:20.987763 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:20.987764 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:20.988388 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:20.988388 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:20.988926 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:20.988927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:20.989260 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:20.989261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:20.989298 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:20.989764 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:20.989768 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:20.991510 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":120,"threads_started":1}
W20260809 19:26:20.991707 31600 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:20.987201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:45137 (local address 127.21.81.254:32983)
0809 19:26:20.987264 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:20.987268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:20.987390 (+ 122us) server_negotiation.cc:407] Connection header received
0809 19:26:20.987446 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:20.987446 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:20.987459 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:20.987504 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:20.987825 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:20.988324 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:20.989002 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:20.989195 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:20.989250 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:20.991606 (+ 2356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.195:45137: BlockingRecv error: recv got EOF from 127.21.81.195:45137 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 19:26:21 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:21 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.193@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:21.748961 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:21.744409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.193:54449)
0809 19:26:21.744591 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:21.744606 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:21.744676 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:21.744842 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:21.744843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:21.745006 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:21.745056 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:21.745058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:21.745714 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:21.745714 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:21.746261 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:21.746263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:21.746568 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:21.746569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:21.746607 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:21.747007 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:21.747011 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:21.748858 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":75,"threads_started":1}
W20260809 19:26:21.748996 31604 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:21.744543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:54449 (local address 127.21.81.254:32983)
0809 19:26:21.744708 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:21.744710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:21.744721 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:26:21.744743 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:21.744743 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:21.744754 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:21.744812 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:21.745121 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:21.745652 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:21.746327 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:21.746508 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:21.746555 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:21.748943 (+ 2388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.193:54449: BlockingRecv error: recv got EOF from 127.21.81.193:54449 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":106,"threads_started":1}
Aug 09 19:26:21 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:26:21 dist-test-slave-njxd krb5kdc[30476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.81.194@KRBTEST.COM for oryx/127.21.81.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:26:21.866969 31605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:21.862318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32983 (local address 127.21.81.194:52023)
0809 19:26:21.862475 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:21.862485 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:21.862544 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:21.862701 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:21.862702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:21.862874 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:21.862925 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:21.862926 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:21.863582 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:21.863583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:21.864118 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:21.864119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:21.864509 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:21.864510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:21.864548 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:21.865022 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:21.865027 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:21.866897 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.81.254:32983: Server oryx/127.21.81.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
W20260809 19:26:21.867101 31604 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:21.862395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:52023 (local address 127.21.81.254:32983)
0809 19:26:21.862497 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:21.862499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:21.862549 (+ 50us) server_negotiation.cc:407] Connection header received
0809 19:26:21.862612 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:21.862613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:21.862624 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:21.862682 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:21.862996 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:21.863498 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:21.864227 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:21.864438 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:21.864500 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:21.867022 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.81.194:52023: BlockingRecv error: recv got EOF from 127.21.81.194:52023 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 19:26:21.899533 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 31101
I20260809 19:26:21.907114 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 31259
I20260809 19:26:21.913676 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 31405
I20260809 19:26:21.921692 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 31027
2026-08-09T19:26:21Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17564 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T19:26:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:26:21Z Disabled control of system clock
I20260809 19:26:21.953864 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:41607
--webserver_interface=127.21.81.254
--webserver_port=0
--builtin_ntp_servers=127.21.81.212:38089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:41607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 19:26:22.035092 31609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:22.035295 31609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:22.035317 31609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:22.036521 31609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:26:22.036563 31609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:22.036576 31609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:22.036588 31609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:26:22.036643 31609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:26:22.038055 31609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:38089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:41607
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:41607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:22.038311 31609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:22.038615 31609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:26:22.043617 31609 server_base.cc:1034] running on GCE node
W20260809 19:26:22.043725 31615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:22.044173 31617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:22.044648 31614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:22.044919 31609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:22.045225 31609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:22.046370 31609 hybrid_clock.cc:648] HybridClock initialized: now 1786303582046355 us; error 38 us; skew 500 ppm
I20260809 19:26:22.047621 31609 webserver.cc:460] Webserver started at http://127.21.81.254:42261/ using document root <none> and password file <none>
I20260809 19:26:22.047880 31609 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:22.047952 31609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:22.048113 31609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:22.049213 31609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "8c8ab277db5b4964bf47452b87f0f1d5"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "f8a5331c4ce2d2a785ca042cce7c36bc"
server_key_iv: "5724c4c4ef01df308a1f0718189c8e83"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.049701 31609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "8c8ab277db5b4964bf47452b87f0f1d5"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "f8a5331c4ce2d2a785ca042cce7c36bc"
server_key_iv: "5724c4c4ef01df308a1f0718189c8e83"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.051074 31609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:26:22.051805 31623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.051971 31609 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:26:22.052074 31609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "8c8ab277db5b4964bf47452b87f0f1d5"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "f8a5331c4ce2d2a785ca042cce7c36bc"
server_key_iv: "5724c4c4ef01df308a1f0718189c8e83"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.052156 31609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:22.072806 31609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:22.073165 31609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:22.073330 31609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:22.077612 31609 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:41607
I20260809 19:26:22.077693 31676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:41607 every 8 connection(s)
I20260809 19:26:22.078047 31609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:26:22.078707 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 31609
I20260809 19:26:22.078827 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:26:22.078896 31678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:22.079052 21831 external_mini_cluster.cc:1428] Setting key d28f193666c8f88dafe02e06e4561c96
I20260809 19:26:22.082288 31678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5: Bootstrap starting.
I20260809 19:26:22.082908 31678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:22.083222 31678 log.cc:824] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5: Log is configured to *not* fsync() on all Append() calls
I20260809 19:26:22.083987 31678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5: No bootstrap required, opened a new log
I20260809 19:26:22.084785 31682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:22.079630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39930 (local address 127.21.81.254:41607)
0809 19:26:22.079855 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:22.079862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:22.079887 (+ 25us) server_negotiation.cc:407] Connection header received
0809 19:26:22.080478 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:22.080486 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:22.080556 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:22.080709 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:22.080914 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:22.081855 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:22.082712 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:22.083155 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:22.083289 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:22.084067 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:22.084071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:22.084071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:26:22.084285 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:22.084327 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:26:22.084341 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:22.084412 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:22.084482 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 19:26:22.084567 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":87,"threads_started":1}
I20260809 19:26:22.086117 31678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8ab277db5b4964bf47452b87f0f1d5" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 41607 } }
I20260809 19:26:22.086251 31678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:22.086274 31678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8ab277db5b4964bf47452b87f0f1d5, State: Initialized, Role: FOLLOWER
I20260809 19:26:22.086412 31678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8ab277db5b4964bf47452b87f0f1d5" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 41607 } }
I20260809 19:26:22.086493 31678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:26:22.086536 31678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:26:22.086582 31678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:22.087136 31678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8ab277db5b4964bf47452b87f0f1d5" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 41607 } }
I20260809 19:26:22.087255 31678 leader_election.cc:302] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c8ab277db5b4964bf47452b87f0f1d5; no voters:
I20260809 19:26:22.087452 31678 leader_election.cc:288] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:26:22.087514 31684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:26:22.087684 31684 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [term 1 LEADER]: Becoming Leader. State: Replica: 8c8ab277db5b4964bf47452b87f0f1d5, State: Running, Role: LEADER
I20260809 19:26:22.087738 31678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:26:22.087838 31684 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8ab277db5b4964bf47452b87f0f1d5" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 41607 } }
I20260809 19:26:22.088189 31685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c8ab277db5b4964bf47452b87f0f1d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8ab277db5b4964bf47452b87f0f1d5" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 41607 } } }
I20260809 19:26:22.088201 31686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c8ab277db5b4964bf47452b87f0f1d5. Latest consensus state: current_term: 1 leader_uuid: "8c8ab277db5b4964bf47452b87f0f1d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8ab277db5b4964bf47452b87f0f1d5" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 41607 } } }
I20260809 19:26:22.088361 31686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:26:22.088547 31685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:26:22.088901 31692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:26:22.089447 31692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:26:22.091073 31692 catalog_manager.cc:1347] Generated new cluster ID: 90317c8879004d8f8c14bfcb563da113
I20260809 19:26:22.091128 31692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:26:22.111398 31692 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:26:22.111996 31692 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:26:22.117213 31692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c8ab277db5b4964bf47452b87f0f1d5: Generated new TSK 0
I20260809 19:26:22.117429 31692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:26:22.122926 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:41607
--builtin_ntp_servers=127.21.81.212:38089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 19:26:22.205502 31704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:22.205796 31704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:22.205837 31704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:22.207039 31704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:22.207108 31704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:22.207146 31704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:26:22.208608 31704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:38089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:41607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:22.208897 31704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:22.209218 31704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:22.215035 31712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:22.215217 31709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:22.215224 31710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:22.215452 31704 server_base.cc:1034] running on GCE node
I20260809 19:26:22.215698 31704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:22.215952 31704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:22.217135 31704 hybrid_clock.cc:648] HybridClock initialized: now 1786303582217127 us; error 38 us; skew 500 ppm
I20260809 19:26:22.218477 31704 webserver.cc:460] Webserver started at http://127.21.81.193:45883/ using document root <none> and password file <none>
I20260809 19:26:22.218741 31704 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:22.218812 31704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:22.218966 31704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:22.220037 31704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "fb2b87c802ef404298b4920c58835b3e"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "d5a3c25f9f0f30af3a03ddff638cc839"
server_key_iv: "bd17ea09b7751c14fdebaab291d381f8"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.220427 31704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "fb2b87c802ef404298b4920c58835b3e"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "d5a3c25f9f0f30af3a03ddff638cc839"
server_key_iv: "bd17ea09b7751c14fdebaab291d381f8"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.221762 31704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:26:22.222472 31718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.222640 31704 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:26:22.222743 31704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "fb2b87c802ef404298b4920c58835b3e"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "d5a3c25f9f0f30af3a03ddff638cc839"
server_key_iv: "bd17ea09b7751c14fdebaab291d381f8"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.222833 31704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:22.236613 31704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:22.236943 31704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:22.237110 31704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:22.237354 31704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:26:22.237744 31704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:22.237803 31704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.237849 31704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:22.237897 31704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.244179 31704 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:46867
I20260809 19:26:22.244228 31838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:46867 every 8 connection(s)
I20260809 19:26:22.244593 31704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:26:22.248682 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 31704
I20260809 19:26:22.248793 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:26:22.248988 21831 external_mini_cluster.cc:1428] Setting key ff89e875b5251a851029f7d549a6e213
I20260809 19:26:22.250149 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:41607
--builtin_ntp_servers=127.21.81.212:38089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 19:26:22.250442 31682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:22.245358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:53253 (local address 127.21.81.254:41607)
0809 19:26:22.245434 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:22.245436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:22.245748 (+ 312us) server_negotiation.cc:407] Connection header received
0809 19:26:22.246339 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:22.246340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:22.246359 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:22.246413 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:22.247132 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:22.247658 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:22.248743 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:22.249002 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:22.249090 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:22.250093 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:22.250094 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:22.250095 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:26:22.250100 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:22.250129 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:26:22.250134 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:22.250199 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:22.250304 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 19:26:22.250349 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 19:26:22.250842 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:22.245339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:41607 (local address 127.21.81.193:53253)
0809 19:26:22.245557 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:22.245653 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:22.246218 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:22.246479 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:22.246483 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:22.246493 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:22.247018 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:22.247023 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:22.247767 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:22.247768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:22.248648 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:22.248651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:22.249106 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:22.249107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:22.249259 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:22.249938 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:26:22.249939 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:22.249981 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:26:22.250004 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:22.250199 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:22.250207 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:22.250210 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:26:22.250565 (+ 355us) client_negotiation.cc:240] Negotiation successful
0809 19:26:22.250681 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260809 19:26:22.251331 31839 heartbeater.cc:347] Connected to a master server at 127.21.81.254:41607
I20260809 19:26:22.251437 31839 heartbeater.cc:464] Registering TS with master...
I20260809 19:26:22.251643 31839 heartbeater.cc:511] Master 127.21.81.254:41607 requested a full tablet report, sending...
I20260809 19:26:22.252251 31640 ts_manager.cc:194] Registered new tserver with Master: fb2b87c802ef404298b4920c58835b3e (127.21.81.193:46867)
I20260809 19:26:22.253216 31640 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.81.193:53253
W20260809 19:26:22.335001 31842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:22.335238 31842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:22.335274 31842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:22.336440 31842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:22.336498 31842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:22.336539 31842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:26:22.337924 31842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:38089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:41607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:22.338187 31842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:22.338496 31842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:22.344928 31849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:22.344939 31847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:22.345239 31842 server_base.cc:1034] running on GCE node
W20260809 19:26:22.345281 31851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:22.345533 31842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:22.345822 31842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:22.346980 31842 hybrid_clock.cc:648] HybridClock initialized: now 1786303582346971 us; error 34 us; skew 500 ppm
I20260809 19:26:22.348230 31842 webserver.cc:460] Webserver started at http://127.21.81.194:37129/ using document root <none> and password file <none>
I20260809 19:26:22.348486 31842 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:22.348553 31842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:22.348699 31842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:22.349852 31842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "2b01dd24031840f28032b8594c625f43"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "3d906fb512fb94c408fa6b6a88aa6f3a"
server_key_iv: "ce5900983959fc3e56c9d0ddf3f42c99"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.350200 31842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "2b01dd24031840f28032b8594c625f43"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "3d906fb512fb94c408fa6b6a88aa6f3a"
server_key_iv: "ce5900983959fc3e56c9d0ddf3f42c99"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.351508 31842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:26:22.352229 31857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.352411 31842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:26:22.352490 31842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "2b01dd24031840f28032b8594c625f43"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "3d906fb512fb94c408fa6b6a88aa6f3a"
server_key_iv: "ce5900983959fc3e56c9d0ddf3f42c99"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.352545 31842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:22.362294 31842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:22.362534 31842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:22.362634 31842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:22.362809 31842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:26:22.363124 31842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:22.363158 31842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.363204 31842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:22.363229 31842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.369273 31842 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:41217
I20260809 19:26:22.369319 31972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:41217 every 8 connection(s)
I20260809 19:26:22.369732 31842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:26:22.374990 31682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:22.370428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:38889 (local address 127.21.81.254:41607)
0809 19:26:22.370474 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:22.370476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:22.370719 (+ 243us) server_negotiation.cc:407] Connection header received
0809 19:26:22.371267 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:22.371269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:22.371282 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:22.371339 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:22.372025 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:22.372527 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:22.373517 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:22.373810 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:22.373885 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:22.374683 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:22.374684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:22.374685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:26:22.374690 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:22.374713 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:26:22.374717 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:22.374789 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:22.374880 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 19:26:22.374908 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:26:22.375226 31975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:22.370411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:41607 (local address 127.21.81.194:38889)
0809 19:26:22.370592 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:22.370626 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:22.371154 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:22.371419 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:22.371423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:22.371438 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:22.371932 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:22.371936 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:22.372628 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:22.372629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:22.373425 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:22.373427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:22.373893 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:22.373895 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:22.374030 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:22.374532 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:26:22.374532 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:22.374573 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:26:22.374595 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:22.374791 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:22.374798 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:22.374801 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:26:22.374881 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 19:26:22.374983 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
I20260809 19:26:22.375713 31973 heartbeater.cc:347] Connected to a master server at 127.21.81.254:41607
I20260809 19:26:22.375790 31973 heartbeater.cc:464] Registering TS with master...
I20260809 19:26:22.375982 31973 heartbeater.cc:511] Master 127.21.81.254:41607 requested a full tablet report, sending...
I20260809 19:26:22.376353 31640 ts_manager.cc:194] Registered new tserver with Master: 2b01dd24031840f28032b8594c625f43 (127.21.81.194:41217)
I20260809 19:26:22.376497 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 31842
I20260809 19:26:22.376581 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:26:22.376781 21831 external_mini_cluster.cc:1428] Setting key 17ba459f38d1beee22d04140a2804510
I20260809 19:26:22.376892 31640 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.81.194:38889
I20260809 19:26:22.377750 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:41607
--builtin_ntp_servers=127.21.81.212:38089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 19:26:22.460794 31976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:22.461001 31976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:22.461023 31976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:22.462263 31976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:22.462306 31976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:22.462330 31976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:26:22.463713 31976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:38089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:41607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:22.463979 31976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:22.464303 31976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:22.469287 31981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:22.469317 31982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:22.469319 31984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:22.469677 31976 server_base.cc:1034] running on GCE node
I20260809 19:26:22.469867 31976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:22.470116 31976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:22.471263 31976 hybrid_clock.cc:648] HybridClock initialized: now 1786303582471250 us; error 33 us; skew 500 ppm
I20260809 19:26:22.472604 31976 webserver.cc:460] Webserver started at http://127.21.81.195:37853/ using document root <none> and password file <none>
I20260809 19:26:22.472913 31976 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:22.472961 31976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:22.473129 31976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:22.474283 31976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "d1e3ebe7025a429abfeeba95781c9693"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "ca7764d7ece775cdc9624da4b7bf1aae"
server_key_iv: "afdfe2a7bb8332b73dd889aef03ea4f3"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.474668 31976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "d1e3ebe7025a429abfeeba95781c9693"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "ca7764d7ece775cdc9624da4b7bf1aae"
server_key_iv: "afdfe2a7bb8332b73dd889aef03ea4f3"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.476006 31976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:26:22.476805 31990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.477051 31976 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:26:22.477144 31976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "d1e3ebe7025a429abfeeba95781c9693"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "ca7764d7ece775cdc9624da4b7bf1aae"
server_key_iv: "afdfe2a7bb8332b73dd889aef03ea4f3"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.477201 31976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:22.486402 31976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:22.486657 31976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:22.486758 31976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:22.486931 31976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:26:22.487246 31976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:22.487295 31976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.487318 31976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:22.487366 31976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.493323 31976 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:40695
I20260809 19:26:22.493396 32108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:40695 every 8 connection(s)
I20260809 19:26:22.493790 31976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:26:22.498747 31682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:22.494528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:47349 (local address 127.21.81.254:41607)
0809 19:26:22.494608 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:22.494611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:22.494866 (+ 255us) server_negotiation.cc:407] Connection header received
0809 19:26:22.495366 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:22.495367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:22.495377 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:22.495431 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:22.496054 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:22.496530 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:22.497457 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:22.497731 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:22.497783 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:22.498527 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:22.498528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:22.498529 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:26:22.498533 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:22.498547 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:26:22.498551 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:22.498590 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:22.498661 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 19:26:22.498682 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 19:26:22.498950 32111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:22.494519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:41607 (local address 127.21.81.195:47349)
0809 19:26:22.494739 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:22.494775 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:22.495284 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:22.495481 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:22.495484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:22.495492 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:22.495985 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:22.495990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:22.496609 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:22.496609 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:22.497385 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:22.497387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:22.497806 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:22.497807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:22.497914 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:22.498396 (+ 482us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:26:22.498396 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:22.498423 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:26:22.498445 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:22.498597 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:22.498600 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:22.498602 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:26:22.498648 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 19:26:22.498723 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":106,"threads_started":1}
I20260809 19:26:22.499552 32109 heartbeater.cc:347] Connected to a master server at 127.21.81.254:41607
I20260809 19:26:22.499639 32109 heartbeater.cc:464] Registering TS with master...
I20260809 19:26:22.499831 32109 heartbeater.cc:511] Master 127.21.81.254:41607 requested a full tablet report, sending...
I20260809 19:26:22.500167 31640 ts_manager.cc:194] Registered new tserver with Master: d1e3ebe7025a429abfeeba95781c9693 (127.21.81.195:40695)
I20260809 19:26:22.500583 31640 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.81.195:47349
I20260809 19:26:22.503156 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 31976
I20260809 19:26:22.503237 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:26:22.503443 21831 external_mini_cluster.cc:1428] Setting key e05d4efdc6cd5fe7e348678e9d953084
I20260809 19:26:22.504319 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 19:26:22.506206 31682 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:22.505654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39932 (local address 127.21.81.254:41607)
0809 19:26:22.505733 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:22.505735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:22.505856 (+ 121us) server_negotiation.cc:407] Connection header received
0809 19:26:22.505917 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:22.505917 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:22.505928 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:22.505983 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:22.506136 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39932: BlockingRecv error: recv got EOF from 127.0.0.1:39932 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 19:26:22.506268 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 31704
I20260809 19:26:22.518332 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 31842
I20260809 19:26:22.524624 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 31976
I20260809 19:26:22.530124 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 31609
2026-08-09T19:26:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (606 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T19:26:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:26:22Z Disabled control of system clock
I20260809 19:26:22.559804 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:43715
--webserver_interface=127.21.81.254
--webserver_port=0
--builtin_ntp_servers=127.21.81.212:34165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:43715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 19:26:22.640877 32121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:22.641145 32121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:22.641186 32121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:22.642361 32121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:26:22.642436 32121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:22.642457 32121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:22.642475 32121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:26:22.642503 32121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:26:22.643843 32121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:34165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:43715
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:43715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:22.644133 32121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:22.644430 32121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:22.649268 32127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:22.649224 32129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:22.649281 32126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:22.649452 32121 server_base.cc:1034] running on GCE node
I20260809 19:26:22.649736 32121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:22.650031 32121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:22.651185 32121 hybrid_clock.cc:648] HybridClock initialized: now 1786303582651163 us; error 41 us; skew 500 ppm
I20260809 19:26:22.652335 32121 webserver.cc:460] Webserver started at http://127.21.81.254:42367/ using document root <none> and password file <none>
I20260809 19:26:22.652572 32121 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:22.652642 32121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:22.652817 32121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:22.653980 32121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "3320db3080594a529a2c08e73861a5d4"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "85583d5f9aa121e1dc60bdb2e4f16390"
server_key_iv: "3b8532c54d24173ca35dd4685012d702"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.654346 32121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "3320db3080594a529a2c08e73861a5d4"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "85583d5f9aa121e1dc60bdb2e4f16390"
server_key_iv: "3b8532c54d24173ca35dd4685012d702"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.655706 32121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:26:22.656483 32135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.656695 32121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:26:22.656791 32121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "3320db3080594a529a2c08e73861a5d4"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "85583d5f9aa121e1dc60bdb2e4f16390"
server_key_iv: "3b8532c54d24173ca35dd4685012d702"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.656868 32121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:22.660403 32121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:22.660686 32121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:22.660831 32121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:22.664989 32121 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:43715
I20260809 19:26:22.665024 32190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:43715 every 8 connection(s)
I20260809 19:26:22.665393 32121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:26:22.666122 32191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:22.668926 32191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4: Bootstrap starting.
I20260809 19:26:22.669480 32191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:22.669824 32191 log.cc:824] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4: Log is configured to *not* fsync() on all Append() calls
I20260809 19:26:22.670481 32191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4: No bootstrap required, opened a new log
I20260809 19:26:22.672029 32191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3320db3080594a529a2c08e73861a5d4" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43715 } }
I20260809 19:26:22.672164 32191 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:22.672186 32191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3320db3080594a529a2c08e73861a5d4, State: Initialized, Role: FOLLOWER
I20260809 19:26:22.672289 32191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3320db3080594a529a2c08e73861a5d4" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43715 } }
I20260809 19:26:22.672374 32191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:26:22.672413 32191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:26:22.672483 32191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:22.673058 32191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3320db3080594a529a2c08e73861a5d4" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43715 } }
I20260809 19:26:22.673177 32191 leader_election.cc:302] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3320db3080594a529a2c08e73861a5d4; no voters:
I20260809 19:26:22.673355 32191 leader_election.cc:288] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:26:22.673410 32195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:26:22.673612 32195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [term 1 LEADER]: Becoming Leader. State: Replica: 3320db3080594a529a2c08e73861a5d4, State: Running, Role: LEADER
I20260809 19:26:22.673655 32191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:26:22.673707 32195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3320db3080594a529a2c08e73861a5d4" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43715 } }
I20260809 19:26:22.674032 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 32121
I20260809 19:26:22.674132 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:26:22.674081 32196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3320db3080594a529a2c08e73861a5d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3320db3080594a529a2c08e73861a5d4" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43715 } } }
I20260809 19:26:22.674331 32196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [sys.catalog]: This master's current role is: LEADER
I20260809 19:26:22.674360 21831 external_mini_cluster.cc:1428] Setting key af721775b08b0bcbf64a9798cedb49ba
I20260809 19:26:22.674353 32197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3320db3080594a529a2c08e73861a5d4. Latest consensus state: current_term: 1 leader_uuid: "3320db3080594a529a2c08e73861a5d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3320db3080594a529a2c08e73861a5d4" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43715 } } }
I20260809 19:26:22.674463 32197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4 [sys.catalog]: This master's current role is: LEADER
I20260809 19:26:22.674721 32200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:26:22.675318 32200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:26:22.676993 32200 catalog_manager.cc:1347] Generated new cluster ID: 3d8cbfdab55a4783aaecdaf4a8461fe8
I20260809 19:26:22.677050 32200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:26:22.677985 32203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:22.674858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48436 (local address 127.21.81.254:43715)
0809 19:26:22.675045 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:22.675056 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:22.676127 (+ 1071us) server_negotiation.cc:407] Connection header received
0809 19:26:22.676341 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:22.676346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:22.676388 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:22.676544 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:22.677537 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:22.677540 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:22.677541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:26:22.677545 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:22.677618 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:26:22.677633 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:22.677694 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:22.677775 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 19:26:22.677833 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260809 19:26:22.684304 32200 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:26:22.684479 32200 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:26:22.692674 32200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3320db3080594a529a2c08e73861a5d4: Generated new TSK 0
I20260809 19:26:22.692848 32200 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:26:22.697104 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43715
--builtin_ntp_servers=127.21.81.212:34165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 19:26:22.780061 32216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:22.780262 32216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:22.780285 32216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:22.781514 32216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:22.781556 32216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:22.781633 32216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:26:22.782948 32216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:34165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:22.783195 32216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:22.783526 32216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:22.789273 32222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:22.789283 32221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:22.789294 32224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:22.789535 32216 server_base.cc:1034] running on GCE node
I20260809 19:26:22.789798 32216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:22.790078 32216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:22.791242 32216 hybrid_clock.cc:648] HybridClock initialized: now 1786303582791224 us; error 34 us; skew 500 ppm
I20260809 19:26:22.792555 32216 webserver.cc:460] Webserver started at http://127.21.81.193:38349/ using document root <none> and password file <none>
I20260809 19:26:22.792811 32216 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:22.792879 32216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:22.793018 32216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:22.794127 32216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "de009ff9babc4871a16d586ac4a0ef53"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "9e13dc57ea05bf31571cc117bdf078ed"
server_key_iv: "97bae448da27c601a25d125a3c834365"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.794488 32216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "de009ff9babc4871a16d586ac4a0ef53"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "9e13dc57ea05bf31571cc117bdf078ed"
server_key_iv: "97bae448da27c601a25d125a3c834365"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.795850 32216 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:26:22.796592 32230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.796806 32216 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:26:22.796903 32216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "de009ff9babc4871a16d586ac4a0ef53"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "9e13dc57ea05bf31571cc117bdf078ed"
server_key_iv: "97bae448da27c601a25d125a3c834365"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.796972 32216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:22.801347 32216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:22.801609 32216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:22.801726 32216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:22.801915 32216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:26:22.802218 32216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:22.802256 32216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.802281 32216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:22.802305 32216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.809051 32216 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:42531
I20260809 19:26:22.809139 32346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:42531 every 8 connection(s)
I20260809 19:26:22.809470 32216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:26:22.812484 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 32216
I20260809 19:26:22.812500 32203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:22.810298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:36667 (local address 127.21.81.254:43715)
0809 19:26:22.810394 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:22.810397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:22.810613 (+ 216us) server_negotiation.cc:407] Connection header received
0809 19:26:22.811092 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:22.811094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:22.811101 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:22.811150 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:22.812225 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:22.812227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:22.812227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:26:22.812231 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:22.812246 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:26:22.812251 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:22.812303 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:22.812416 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 19:26:22.812431 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 19:26:22.812630 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:26:22.812690 32349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:22.810250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:43715 (local address 127.21.81.193:36667)
0809 19:26:22.810472 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:22.810505 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:22.810987 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:22.811214 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:22.811220 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:22.811233 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:22.812073 (+ 840us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:26:22.812074 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:22.812107 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:26:22.812135 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:22.812320 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:22.812325 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:22.812328 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:26:22.812411 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:26:22.812471 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":91,"threads_started":1}
I20260809 19:26:22.812871 21831 external_mini_cluster.cc:1428] Setting key b439f67dc02f951b7d36eb3d97da52c7
I20260809 19:26:22.813234 32347 heartbeater.cc:347] Connected to a master server at 127.21.81.254:43715
I20260809 19:26:22.813318 32347 heartbeater.cc:464] Registering TS with master...
I20260809 19:26:22.813496 32347 heartbeater.cc:511] Master 127.21.81.254:43715 requested a full tablet report, sending...
I20260809 19:26:22.813927 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43715
--builtin_ntp_servers=127.21.81.212:34165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 19:26:22.814114 32153 ts_manager.cc:194] Registered new tserver with Master: de009ff9babc4871a16d586ac4a0ef53 (127.21.81.193:42531)
W20260809 19:26:22.894670 32350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:22.894860 32350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:22.894881 32350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:22.896034 32350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:22.896071 32350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:22.896102 32350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:26:22.897363 32350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:34165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:22.897658 32350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:22.897953 32350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:22.902832 32356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:22.902840 32359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:22.902870 32357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:22.903028 32350 server_base.cc:1034] running on GCE node
I20260809 19:26:22.903194 32350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:22.903438 32350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:22.904603 32350 hybrid_clock.cc:648] HybridClock initialized: now 1786303582904591 us; error 35 us; skew 500 ppm
I20260809 19:26:22.905937 32350 webserver.cc:460] Webserver started at http://127.21.81.194:43719/ using document root <none> and password file <none>
I20260809 19:26:22.906193 32350 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:22.906265 32350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:22.906419 32350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:22.907428 32350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "12f1997bf86744a3a4953def6c8300aa"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "352a6d7c610562d73b0062a781848f78"
server_key_iv: "5d84d80f18ed96a9b2c38da73423735b"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.907806 32350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "12f1997bf86744a3a4953def6c8300aa"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "352a6d7c610562d73b0062a781848f78"
server_key_iv: "5d84d80f18ed96a9b2c38da73423735b"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.909247 32350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:26:22.910037 32365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.910209 32350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.910310 32350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "12f1997bf86744a3a4953def6c8300aa"
format_stamp: "Formatted at 2026-08-09 19:26:22 on dist-test-slave-njxd"
server_key: "352a6d7c610562d73b0062a781848f78"
server_key_iv: "5d84d80f18ed96a9b2c38da73423735b"
server_key_version: "encryptionkey@0"
I20260809 19:26:22.910379 32350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:22.913532 32350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:22.913791 32350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:22.913931 32350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:22.914150 32350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:26:22.914440 32350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:22.914490 32350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.914527 32350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:22.914552 32350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:22.920668 32350 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:42783
I20260809 19:26:22.920725 32483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:42783 every 8 connection(s)
I20260809 19:26:22.921001 32350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:26:22.923839 32203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:22.921749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:48839 (local address 127.21.81.254:43715)
0809 19:26:22.921820 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:22.921823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:22.922043 (+ 220us) server_negotiation.cc:407] Connection header received
0809 19:26:22.922472 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:22.922473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:22.922479 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:22.922530 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:22.923640 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:22.923642 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:22.923642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:26:22.923645 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:22.923660 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:26:22.923664 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:22.923709 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:22.923773 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 19:26:22.923785 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:26:22.924034 32486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:22.921756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:43715 (local address 127.21.81.194:48839)
0809 19:26:22.921922 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:22.921953 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:22.922387 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:22.922583 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:22.922587 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:22.922595 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:22.923483 (+ 888us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:26:22.923483 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:22.923510 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:26:22.923536 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:22.923709 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:22.923711 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:22.923713 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:26:22.923764 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:26:22.923810 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260809 19:26:22.924533 32484 heartbeater.cc:347] Connected to a master server at 127.21.81.254:43715
I20260809 19:26:22.924612 32484 heartbeater.cc:464] Registering TS with master...
I20260809 19:26:22.924778 32484 heartbeater.cc:511] Master 127.21.81.254:43715 requested a full tablet report, sending...
I20260809 19:26:22.925149 32153 ts_manager.cc:194] Registered new tserver with Master: 12f1997bf86744a3a4953def6c8300aa (127.21.81.194:42783)
I20260809 19:26:22.928968 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 32350
I20260809 19:26:22.929080 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:26:22.929278 21831 external_mini_cluster.cc:1428] Setting key 1f0047564b2f48fd112a488dabaea552
I20260809 19:26:22.930485 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43715
--builtin_ntp_servers=127.21.81.212:34165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 19:26:23.012373 32487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:23.012634 32487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:23.012663 32487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:23.013933 32487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:23.013971 32487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:23.014019 32487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:26:23.015357 32487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:34165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:23.015605 32487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:23.015910 32487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:23.020645 32492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:23.020680 32493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:23.020670 32495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:23.020867 32487 server_base.cc:1034] running on GCE node
I20260809 19:26:23.021100 32487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:23.021369 32487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:23.022524 32487 hybrid_clock.cc:648] HybridClock initialized: now 1786303583022498 us; error 56 us; skew 500 ppm
I20260809 19:26:23.023669 32487 webserver.cc:460] Webserver started at http://127.21.81.195:32881/ using document root <none> and password file <none>
I20260809 19:26:23.023914 32487 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:23.023978 32487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:23.024112 32487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:23.025094 32487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "3bd63297e0474b4f8211e24699edc3d2"
format_stamp: "Formatted at 2026-08-09 19:26:23 on dist-test-slave-njxd"
server_key: "b48fd9a3ded2ddff8c1a336ae0b3ba23"
server_key_iv: "e20f5bcafad8aecdc3f546b21ab12380"
server_key_version: "encryptionkey@0"
I20260809 19:26:23.025466 32487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "3bd63297e0474b4f8211e24699edc3d2"
format_stamp: "Formatted at 2026-08-09 19:26:23 on dist-test-slave-njxd"
server_key: "b48fd9a3ded2ddff8c1a336ae0b3ba23"
server_key_iv: "e20f5bcafad8aecdc3f546b21ab12380"
server_key_version: "encryptionkey@0"
I20260809 19:26:23.026777 32487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:26:23.027498 32501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:23.027686 32487 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:26:23.027786 32487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "3bd63297e0474b4f8211e24699edc3d2"
format_stamp: "Formatted at 2026-08-09 19:26:23 on dist-test-slave-njxd"
server_key: "b48fd9a3ded2ddff8c1a336ae0b3ba23"
server_key_iv: "e20f5bcafad8aecdc3f546b21ab12380"
server_key_version: "encryptionkey@0"
I20260809 19:26:23.027864 32487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:23.031106 32487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:23.031366 32487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:23.031502 32487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:23.031697 32487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:26:23.031985 32487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:23.032035 32487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:23.032071 32487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:23.032099 32487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:23.037495 32487 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:34229
I20260809 19:26:23.037619 32615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:34229 every 8 connection(s)
I20260809 19:26:23.037915 32487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:26:23.040694 32203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:23.038735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:58329 (local address 127.21.81.254:43715)
0809 19:26:23.038807 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:23.038810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:23.038953 (+ 143us) server_negotiation.cc:407] Connection header received
0809 19:26:23.039378 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:23.039379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:23.039383 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:23.039455 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:23.040505 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:23.040506 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:23.040506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:26:23.040511 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:23.040526 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:26:23.040531 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:23.040576 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:23.040628 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 19:26:23.040640 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 19:26:23.040895 32618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:23.038638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:43715 (local address 127.21.81.195:58329)
0809 19:26:23.038821 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:23.038870 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:23.039298 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:23.039499 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:23.039502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:23.039509 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:23.040363 (+ 854us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:26:23.040364 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:23.040389 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:26:23.040415 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:23.040580 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:23.040582 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:23.040584 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:26:23.040627 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 19:26:23.040671 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260809 19:26:23.041426 32616 heartbeater.cc:347] Connected to a master server at 127.21.81.254:43715
I20260809 19:26:23.041539 32616 heartbeater.cc:464] Registering TS with master...
I20260809 19:26:23.041709 32616 heartbeater.cc:511] Master 127.21.81.254:43715 requested a full tablet report, sending...
I20260809 19:26:23.042052 32153 ts_manager.cc:194] Registered new tserver with Master: 3bd63297e0474b4f8211e24699edc3d2 (127.21.81.195:34229)
I20260809 19:26:23.045814 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 32487
I20260809 19:26:23.045909 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:26:23.046109 21831 external_mini_cluster.cc:1428] Setting key 9ea5f389f4f8f7d5a6301940ca999009
I20260809 19:26:23.047003 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 19:26:23.048836 32203 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:26:23.048354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48442 (local address 127.21.81.254:43715)
0809 19:26:23.048413 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:23.048415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:23.048534 (+ 119us) server_negotiation.cc:407] Connection header received
0809 19:26:23.048594 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:23.048595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:23.048600 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:23.048653 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:23.048769 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48442: BlockingRecv error: recv got EOF from 127.0.0.1:48442 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:26:23.048939 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 32216
I20260809 19:26:23.057840 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 32350
I20260809 19:26:23.067636 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 32487
I20260809 19:26:23.076121 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 32121
2026-08-09T19:26:23Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (546 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:26:23 dist-test-slave-njxd krb5kdc[32627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:26:23 dist-test-slave-njxd krb5kdc[32627](info): set up 2 sockets
Aug 09 19:26:23 dist-test-slave-njxd krb5kdc[32627](info): commencing operation
krb5kdc: starting...
W20260809 19:26:24.155000 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:26:24 dist-test-slave-njxd krb5kdc[32627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:26:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:26:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:26:24.279882 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:40007
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:40007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:26:24.364389 32640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:24.364600 32640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:24.364622 32640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:24.365734 32640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:26:24.365772 32640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:24.365785 32640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:24.365797 32640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:26:24.365808 32640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:26:24.367237 32640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:40007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:40007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:24.367429 32640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:24.367643 32640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:24.372565 32645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:24.372619 32647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:24.372826 32640 server_base.cc:1034] running on GCE node
W20260809 19:26:24.372557 32649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:24.373131 32640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:24.373436 32640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:24.374605 32640 hybrid_clock.cc:648] HybridClock initialized: now 1786303584374581 us; error 37 us; skew 500 ppm
Aug 09 19:26:24 dist-test-slave-njxd krb5kdc[32627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303584, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:24.377089 32640 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:26:24.377564 32640 webserver.cc:460] Webserver started at http://127.21.81.254:39277/ using document root <none> and password file <none>
I20260809 19:26:24.377858 32640 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:24.377928 32640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:24.378075 32640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:24.379130 32640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "05eb8c7a162f462b9b0302be6ed0b383"
format_stamp: "Formatted at 2026-08-09 19:26:24 on dist-test-slave-njxd"
server_key: "80853c96df96eac934ef94129a20ccf3"
server_key_iv: "d802edacdce8b4ad2fe1ffba9f7a5058"
server_key_version: "encryptionkey@0"
I20260809 19:26:24.379501 32640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "05eb8c7a162f462b9b0302be6ed0b383"
format_stamp: "Formatted at 2026-08-09 19:26:24 on dist-test-slave-njxd"
server_key: "80853c96df96eac934ef94129a20ccf3"
server_key_iv: "d802edacdce8b4ad2fe1ffba9f7a5058"
server_key_version: "encryptionkey@0"
I20260809 19:26:24.380832 32640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:26:24.381547 32656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:24.381767 32640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:26:24.381879 32640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "05eb8c7a162f462b9b0302be6ed0b383"
format_stamp: "Formatted at 2026-08-09 19:26:24 on dist-test-slave-njxd"
server_key: "80853c96df96eac934ef94129a20ccf3"
server_key_iv: "d802edacdce8b4ad2fe1ffba9f7a5058"
server_key_version: "encryptionkey@0"
I20260809 19:26:24.381959 32640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:24.394960 32640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:24.396811 32640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:24.396982 32640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:24.401371 32640 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:40007
I20260809 19:26:24.401413 32709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:40007 every 8 connection(s)
I20260809 19:26:24.401868 32640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:26:24.402946 32710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:24.405088 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 32640
I20260809 19:26:24.405212 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:26:24.405296 32710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383: Bootstrap starting.
I20260809 19:26:24.405447 21831 external_mini_cluster.cc:1428] Setting key aaaf16bcf5bcc0e31ec5be38b00ae6d9
I20260809 19:26:24.406102 32710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:24.406445 32710 log.cc:824] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383: Log is configured to *not* fsync() on all Append() calls
I20260809 19:26:24.407248 32710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383: No bootstrap required, opened a new log
I20260809 19:26:24.409519 32710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05eb8c7a162f462b9b0302be6ed0b383" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40007 } }
I20260809 19:26:24.409719 32710 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:24.409775 32710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05eb8c7a162f462b9b0302be6ed0b383, State: Initialized, Role: FOLLOWER
I20260809 19:26:24.409904 32710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05eb8c7a162f462b9b0302be6ed0b383" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40007 } }
I20260809 19:26:24.410012 32710 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:26:24.410061 32710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:26:24.410112 32710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:24.410777 32710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05eb8c7a162f462b9b0302be6ed0b383" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40007 } }
I20260809 19:26:24.410913 32710 leader_election.cc:302] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05eb8c7a162f462b9b0302be6ed0b383; no voters:
I20260809 19:26:24.411073 32710 leader_election.cc:288] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:26:24.411217 32716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:26:24.411375 32710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:26:24.411432 32716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [term 1 LEADER]: Becoming Leader. State: Replica: 05eb8c7a162f462b9b0302be6ed0b383, State: Running, Role: LEADER
I20260809 19:26:24.411569 32716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05eb8c7a162f462b9b0302be6ed0b383" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40007 } }
I20260809 19:26:24.412003 32718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05eb8c7a162f462b9b0302be6ed0b383. Latest consensus state: current_term: 1 leader_uuid: "05eb8c7a162f462b9b0302be6ed0b383" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05eb8c7a162f462b9b0302be6ed0b383" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40007 } } }
I20260809 19:26:24.412091 32718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [sys.catalog]: This master's current role is: LEADER
I20260809 19:26:24.412338 32717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05eb8c7a162f462b9b0302be6ed0b383" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05eb8c7a162f462b9b0302be6ed0b383" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40007 } } }
I20260809 19:26:24.412609 32717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383 [sys.catalog]: This master's current role is: LEADER
I20260809 19:26:24.412781 32723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 19:26:24 dist-test-slave-njxd krb5kdc[32627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:26:24.413300 32723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:26:24.415594 32723 catalog_manager.cc:1347] Generated new cluster ID: 4d4ec0c4fe34412092d5eb59a5de63dd
I20260809 19:26:24.415692 32723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:26:24.417482 32713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.406020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57012 (local address 127.21.81.254:40007)
0809 19:26:24.406302 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:24.406310 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:24.406334 (+ 24us) server_negotiation.cc:407] Connection header received
0809 19:26:24.406933 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:24.406944 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:24.407002 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:24.407120 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:24.407944 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.408833 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.409954 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.410463 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.410546 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.413815 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:24.413819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:24.413819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:24.413831 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:24.415935 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:24.416307 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:24.416310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:24.416310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:24.416409 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:24.416623 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:24.416624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:24.416625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:24.416863 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:24.417046 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:24.417113 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 19:26:24.417268 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":106,"threads_started":1}
I20260809 19:26:24.426716 32723 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:26:24.427222 32723 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:26:24.431002 32723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383: Generated new TSK 0
I20260809 19:26:24.431200 32723 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:26:24.464576 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:40007
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:26:24.565426 32737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:24.565709 32737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:24.565753 32737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:24.566928 32737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:24.566993 32737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:24.567039 32737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:26:24.568538 32737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:40007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:24.568830 32737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:24.569082 32737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:24.573926 32746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:24.573937 32744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:24.573926 32743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:24.574122 32737 server_base.cc:1034] running on GCE node
I20260809 19:26:24.574350 32737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:24.574649 32737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:24.575795 32737 hybrid_clock.cc:648] HybridClock initialized: now 1786303584575784 us; error 29 us; skew 500 ppm
Aug 09 19:26:24 dist-test-slave-njxd krb5kdc[32627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303584, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:24.578277 32737 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:26:24.578645 32737 webserver.cc:460] Webserver started at http://127.21.81.193:42091/ using document root <none> and password file <none>
I20260809 19:26:24.578830 32737 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:24.578881 32737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:24.578967 32737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:24.579963 32737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "5499a666793446f0a25ab69684366c33"
format_stamp: "Formatted at 2026-08-09 19:26:24 on dist-test-slave-njxd"
server_key: "03ed82b5dd09a7d129737378742b487c"
server_key_iv: "d4d4e8ccf3729ac2a375b8deba43be78"
server_key_version: "encryptionkey@0"
I20260809 19:26:24.580261 32737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "5499a666793446f0a25ab69684366c33"
format_stamp: "Formatted at 2026-08-09 19:26:24 on dist-test-slave-njxd"
server_key: "03ed82b5dd09a7d129737378742b487c"
server_key_iv: "d4d4e8ccf3729ac2a375b8deba43be78"
server_key_version: "encryptionkey@0"
I20260809 19:26:24.581496 32737 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:26:24.582230 32753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:24.582418 32737 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:26:24.582506 32737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "5499a666793446f0a25ab69684366c33"
format_stamp: "Formatted at 2026-08-09 19:26:24 on dist-test-slave-njxd"
server_key: "03ed82b5dd09a7d129737378742b487c"
server_key_iv: "d4d4e8ccf3729ac2a375b8deba43be78"
server_key_version: "encryptionkey@0"
I20260809 19:26:24.582580 32737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:24.590845 32737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:24.592552 32737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:24.592705 32737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:24.592911 32737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:26:24.593214 32737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:24.593272 32737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:24.593329 32737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:24.593358 32737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:24.599431 32737 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:36287
I20260809 19:26:24.599529 404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:36287 every 8 connection(s)
I20260809 19:26:24.599807 32737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:26:24.600778 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 32737
I20260809 19:26:24.600891 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:26:24.601102 21831 external_mini_cluster.cc:1428] Setting key 29c7a89ff7238dfb035959525e016256
Aug 09 19:26:24 dist-test-slave-njxd krb5kdc[32627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303584, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:26:24.610460 32713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.600502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:50457 (local address 127.21.81.254:40007)
0809 19:26:24.600570 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:24.600573 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:24.600859 (+ 286us) server_negotiation.cc:407] Connection header received
0809 19:26:24.601539 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:24.601540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:24.601551 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:24.601682 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:24.602997 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.603492 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.604627 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.604968 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.605062 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.607272 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:24.607274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:24.607274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:24.607284 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:24.608950 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:24.609346 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:24.609348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:24.609348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:24.609419 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:24.609723 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:24.609724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:24.609725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:24.609871 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:24.609955 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:24.610235 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 19:26:24.610343 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:26:24.610481 407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.600477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:40007 (local address 127.21.81.193:50457)
0809 19:26:24.600682 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:24.600742 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:24.601431 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:24.601764 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:24.601768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:24.602599 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:24.602894 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.602901 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.603594 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.603595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.604524 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.604527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.605065 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.605066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.605167 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.605774 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:24.605783 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:24.607173 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:24.609037 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:24.609044 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:24.609046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:24.609266 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:24.609493 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:24.609494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:24.609494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:24.609655 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:24.609950 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:24.609955 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:24.610100 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 19:26:24.610183 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:26:24.610307 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":113,"threads_started":1}
I20260809 19:26:24.611070 405 heartbeater.cc:347] Connected to a master server at 127.21.81.254:40007
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
I20260809 19:26:24.611187 405 heartbeater.cc:464] Registering TS with master...
I20260809 19:26:24.611390 405 heartbeater.cc:511] Master 127.21.81.254:40007 requested a full tablet report, sending...
I20260809 19:26:24.611982 32674 ts_manager.cc:194] Registered new tserver with Master: 5499a666793446f0a25ab69684366c33 (127.21.81.193:36287)
I20260809 19:26:24.612735 32674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:50457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:26:24.629282 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:40007
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:26:24.713809 410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:24.714030 410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:24.714066 410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:24.715227 410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:24.715297 410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:24.715345 410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:26:24.716815 410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:40007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:24.717077 410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:24.717334 410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:24.722294 416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:24.722362 415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:24.722393 410 server_base.cc:1034] running on GCE node
W20260809 19:26:24.722329 418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:24.722680 410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:24.722925 410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:24.724089 410 hybrid_clock.cc:648] HybridClock initialized: now 1786303584724074 us; error 42 us; skew 500 ppm
Aug 09 19:26:24 dist-test-slave-njxd krb5kdc[32627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303584, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:24.726761 410 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:26:24.727133 410 webserver.cc:460] Webserver started at http://127.21.81.194:37731/ using document root <none> and password file <none>
I20260809 19:26:24.727329 410 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:24.727382 410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:24.727468 410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:24.728439 410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "e24373b270894e3bb6e91e8550465a46"
format_stamp: "Formatted at 2026-08-09 19:26:24 on dist-test-slave-njxd"
server_key: "bff7ce9c9f6741e53e04c35bd9dd6f6a"
server_key_iv: "cf2fe606c1234a45e4eaf185d3e60155"
server_key_version: "encryptionkey@0"
I20260809 19:26:24.728746 410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "e24373b270894e3bb6e91e8550465a46"
format_stamp: "Formatted at 2026-08-09 19:26:24 on dist-test-slave-njxd"
server_key: "bff7ce9c9f6741e53e04c35bd9dd6f6a"
server_key_iv: "cf2fe606c1234a45e4eaf185d3e60155"
server_key_version: "encryptionkey@0"
I20260809 19:26:24.730041 410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:26:24.730754 425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:24.730944 410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:26:24.731017 410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "e24373b270894e3bb6e91e8550465a46"
format_stamp: "Formatted at 2026-08-09 19:26:24 on dist-test-slave-njxd"
server_key: "bff7ce9c9f6741e53e04c35bd9dd6f6a"
server_key_iv: "cf2fe606c1234a45e4eaf185d3e60155"
server_key_version: "encryptionkey@0"
I20260809 19:26:24.731113 410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:24.747920 410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:24.750308 410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:24.750488 410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:24.750751 410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:26:24.751067 410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:24.751102 410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:24.751125 410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:24.751174 410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:24.757025 410 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:45045
I20260809 19:26:24.757123 542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:45045 every 8 connection(s)
I20260809 19:26:24.757400 410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:26:24 dist-test-slave-njxd krb5kdc[32627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303584, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:26:24.764811 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 410
I20260809 19:26:24.764899 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:26:24.765125 21831 external_mini_cluster.cc:1428] Setting key 95dde4b6b54d6bcf142ee971f3f74540
I20260809 19:26:24.766484 32713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.758201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:33839 (local address 127.21.81.254:40007)
0809 19:26:24.758248 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:24.758250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:24.758429 (+ 179us) server_negotiation.cc:407] Connection header received
0809 19:26:24.758925 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:24.758926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:24.758935 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:24.758997 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:24.759873 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.760366 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.761300 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.761477 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.761532 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.763704 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:24.763705 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:24.763705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:24.763711 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:24.764927 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:24.765425 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:24.765426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:24.765427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:24.765492 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:24.765793 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:24.765794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:24.765794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:24.765930 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:24.766005 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:24.766268 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 19:26:24.766356 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:26:24.766491 545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.758148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:40007 (local address 127.21.81.194:33839)
0809 19:26:24.758309 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:24.758352 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:24.758835 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:24.759047 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:24.759050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:24.759608 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:24.759793 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.759798 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.760446 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.760447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.761224 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.761226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.761635 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.761636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.761746 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.762260 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:24.762268 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:24.763618 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:24.765029 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:24.765036 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:24.765038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:24.765324 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:24.765587 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:24.765588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:24.765588 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:24.765721 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:24.766004 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:24.766010 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:24.766130 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 19:26:24.766216 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 19:26:24.766337 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260809 19:26:24.767223 543 heartbeater.cc:347] Connected to a master server at 127.21.81.254:40007
I20260809 19:26:24.767338 543 heartbeater.cc:464] Registering TS with master...
I20260809 19:26:24.767515 543 heartbeater.cc:511] Master 127.21.81.254:40007 requested a full tablet report, sending...
I20260809 19:26:24.767962 32674 ts_manager.cc:194] Registered new tserver with Master: e24373b270894e3bb6e91e8550465a46 (127.21.81.194:45045)
I20260809 19:26:24.768488 32674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:33839
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:26:24.792230 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:40007
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:26:24.873538 548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:26:24.873823 548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:26:24.873848 548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:26:24.874964 548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:26:24.875000 548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:26:24.875025 548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:26:24.876395 548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:40007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:26:24.876570 548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:26:24.876748 548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:26:24.881615 554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:24.881675 553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:26:24.881783 556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:26:24.882027 548 server_base.cc:1034] running on GCE node
I20260809 19:26:24.882216 548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:26:24.882403 548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:26:24.883534 548 hybrid_clock.cc:648] HybridClock initialized: now 1786303584883524 us; error 27 us; skew 500 ppm
Aug 09 19:26:24 dist-test-slave-njxd krb5kdc[32627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303584, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:26:24.890350 548 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:26:24.890807 548 webserver.cc:460] Webserver started at http://127.21.81.195:38469/ using document root <none> and password file <none>
I20260809 19:26:24.891054 548 fs_manager.cc:359] Metadata directory not provided
I20260809 19:26:24.891100 548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:26:24.891230 548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:26:24.892270 548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "17c92e258d644f448c9684ec7fc3d524"
format_stamp: "Formatted at 2026-08-09 19:26:24 on dist-test-slave-njxd"
server_key: "aee06dea1e3a3275673940fc1979af6e"
server_key_iv: "86ba0485ede945562533b7196fc39e34"
server_key_version: "encryptionkey@0"
I20260809 19:26:24.892629 548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "17c92e258d644f448c9684ec7fc3d524"
format_stamp: "Formatted at 2026-08-09 19:26:24 on dist-test-slave-njxd"
server_key: "aee06dea1e3a3275673940fc1979af6e"
server_key_iv: "86ba0485ede945562533b7196fc39e34"
server_key_version: "encryptionkey@0"
I20260809 19:26:24.894151 548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:26:24.894935 564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:24.895114 548 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:26:24.895206 548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "17c92e258d644f448c9684ec7fc3d524"
format_stamp: "Formatted at 2026-08-09 19:26:24 on dist-test-slave-njxd"
server_key: "aee06dea1e3a3275673940fc1979af6e"
server_key_iv: "86ba0485ede945562533b7196fc39e34"
server_key_version: "encryptionkey@0"
I20260809 19:26:24.895290 548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:26:24.913401 548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:26:24.915938 548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:26:24.916109 548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:26:24.916354 548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:26:24.916667 548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:26:24.916720 548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:24.916762 548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:26:24.916790 548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:26:24.923223 548 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:39157
I20260809 19:26:24.923266 679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:39157 every 8 connection(s)
I20260809 19:26:24.923570 548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:26:24.927474 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 548
I20260809 19:26:24.927568 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:26:24.927752 21831 external_mini_cluster.cc:1428] Setting key 84ca47c03410185f4d136ad633538544
Aug 09 19:26:24 dist-test-slave-njxd krb5kdc[32627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303584, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:26:24.932754 32713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.924275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:59019 (local address 127.21.81.254:40007)
0809 19:26:24.924358 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:24.924360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:24.924535 (+ 175us) server_negotiation.cc:407] Connection header received
0809 19:26:24.925001 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:24.925002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:24.925012 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:24.925063 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:24.926016 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.926490 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.927448 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.927777 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.927852 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.930183 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:24.930184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:24.930185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:24.930191 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:24.931659 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:24.932007 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:24.932008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:24.932008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:24.932059 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:24.932270 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:24.932271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:24.932271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:24.932365 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:24.932414 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:24.932632 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 19:26:24.932693 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 19:26:24.932909 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.924231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:40007 (local address 127.21.81.195:59019)
0809 19:26:24.924418 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:24.924453 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:24.924924 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:24.925121 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:24.925124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:24.925746 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:26:24.925941 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.925946 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.926568 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.926569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.927336 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.927338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.927894 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.927895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.928046 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.928504 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:26:24.928513 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:26:24.930093 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:26:24.931730 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:24.931736 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:24.931737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:24.931941 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:24.932116 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:26:24.932116 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:26:24.932116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:26:24.932217 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:26:24.932424 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:26:24.932428 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:26:24.932531 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 19:26:24.932608 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 19:26:24.932721 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":103,"threads_started":1}
I20260809 19:26:24.933333 680 heartbeater.cc:347] Connected to a master server at 127.21.81.254:40007
I20260809 19:26:24.933439 680 heartbeater.cc:464] Registering TS with master...
I20260809 19:26:24.933660 680 heartbeater.cc:511] Master 127.21.81.254:40007 requested a full tablet report, sending...
I20260809 19:26:24.934056 32674 ts_manager.cc:194] Registered new tserver with Master: 17c92e258d644f448c9684ec7fc3d524 (127.21.81.195:39157)
I20260809 19:26:24.934425 32674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:59019
I20260809 19:26:24.939761 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:26:24.947083 32713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.941013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57020 (local address 127.21.81.254:40007)
0809 19:26:24.941188 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:24.941190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:24.941265 (+ 75us) server_negotiation.cc:407] Connection header received
0809 19:26:24.941338 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:24.941339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:24.941348 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:24.941415 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:26:24.942025 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.942478 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.943322 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.943524 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.943634 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.944782 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:26:24.944784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:26:24.944784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:26:24.944791 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:26:24.945930 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:24.946284 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:24.946285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:24.946285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:24.946343 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:26:24.946548 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:26:24.946549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:26:24.946549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:26:24.946699 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:26:24.946809 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:24.946910 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 19:26:24.946987 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260809 19:26:24.948685 32674 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:26:24.950371 32674 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:26:24.956825 692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.952560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:45045 (local address 127.0.0.1:37050)
0809 19:26:24.952957 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:24.952983 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:24.953023 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:24.953334 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:24.953335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:24.953337 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:26:24.953398 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.953400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.954449 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.954450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.955788 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.955790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.956502 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.956503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.956583 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.956596 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:26:24.956685 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 19:26:24.956736 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":2,"thread_start_us":91,"threads_started":1}
I20260809 19:26:24.956985 691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.952560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:39157 (local address 127.0.0.1:57078)
0809 19:26:24.952737 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:24.952782 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:24.952847 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:24.953285 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:24.953288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:24.953291 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:26:24.953363 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.953365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.954817 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.954818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.955898 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.955900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.956681 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.956682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.956738 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.956741 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:26:24.956792 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:26:24.956822 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":26,"thread_start_us":45,"threads_started":1}
I20260809 19:26:24.957082 697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.952560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:36287 (local address 127.0.0.1:42718)
0809 19:26:24.953163 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:24.953175 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:24.953233 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:24.953516 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:24.953517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:24.953519 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:26:24.953557 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.953558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.954866 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.954867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.955774 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.955777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.956859 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.956859 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.956923 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.956926 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:26:24.956984 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 19:26:24.957026 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":452,"thread_start_us":91,"threads_started":1}
I20260809 19:26:24.957195 696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.952839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37050 (local address 127.21.81.194:45045)
0809 19:26:24.953160 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:24.953167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:24.953176 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:26:24.953215 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:24.953217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:24.953233 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:24.953335 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:26:24.953494 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.954314 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.955894 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.956405 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.956489 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.956549 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:24.957048 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 19:26:24.957095 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":123,"threads_started":1}
I20260809 19:26:24.957321 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.952669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42718 (local address 127.21.81.193:36287)
0809 19:26:24.953221 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:24.953226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:24.953238 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:26:24.953305 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:24.953307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:24.953322 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:24.953398 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:26:24.953683 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.954777 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.955870 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.956595 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.956672 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.956729 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:24.957182 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 19:26:24.957237 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":104,"threads_started":1}
I20260809 19:26:24.957795 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.952872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57078 (local address 127.21.81.195:39157)
0809 19:26:24.953067 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:24.953072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:24.953083 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:26:24.953125 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:24.953127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:24.953143 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:24.953489 (+ 346us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:26:24.953566 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.954704 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.955968 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.956596 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.956672 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.956723 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:24.956804 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 19:26:24.956835 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":87,"threads_started":1}
I20260809 19:26:24.957962 476 tablet_service.cc:1467] Processing CreateTablet for tablet e0ba02f33ad141caaa3eeab5aa4402cc (DEFAULT_TABLE table=test-table [id=d795f3295144486a9c88680941397141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:26:24.958096 335 tablet_service.cc:1467] Processing CreateTablet for tablet e0ba02f33ad141caaa3eeab5aa4402cc (DEFAULT_TABLE table=test-table [id=d795f3295144486a9c88680941397141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:26:24.958338 476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0ba02f33ad141caaa3eeab5aa4402cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:24.958396 335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0ba02f33ad141caaa3eeab5aa4402cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:24.958500 613 tablet_service.cc:1467] Processing CreateTablet for tablet e0ba02f33ad141caaa3eeab5aa4402cc (DEFAULT_TABLE table=test-table [id=d795f3295144486a9c88680941397141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:26:24.958751 613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0ba02f33ad141caaa3eeab5aa4402cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:26:24.961226 698 tablet_bootstrap.cc:492] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524: Bootstrap starting.
I20260809 19:26:24.961778 700 tablet_bootstrap.cc:492] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33: Bootstrap starting.
I20260809 19:26:24.961771 699 tablet_bootstrap.cc:492] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46: Bootstrap starting.
I20260809 19:26:24.961961 698 tablet_bootstrap.cc:654] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:24.962409 699 tablet_bootstrap.cc:654] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:24.962481 700 tablet_bootstrap.cc:654] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33: Neither blocks nor log segments found. Creating new log.
I20260809 19:26:24.962637 698 log.cc:824] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524: Log is configured to *not* fsync() on all Append() calls
I20260809 19:26:24.963064 699 log.cc:824] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46: Log is configured to *not* fsync() on all Append() calls
I20260809 19:26:24.963086 700 log.cc:824] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33: Log is configured to *not* fsync() on all Append() calls
I20260809 19:26:24.963560 698 tablet_bootstrap.cc:492] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524: No bootstrap required, opened a new log
I20260809 19:26:24.963649 698 ts_tablet_manager.cc:1397] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:26:24.963840 700 tablet_bootstrap.cc:492] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33: No bootstrap required, opened a new log
I20260809 19:26:24.963910 700 ts_tablet_manager.cc:1397] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:26:24.964115 699 tablet_bootstrap.cc:492] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46: No bootstrap required, opened a new log
I20260809 19:26:24.964182 699 ts_tablet_manager.cc:1397] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:26:24.965493 699 raft_consensus.cc:348] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5499a666793446f0a25ab69684366c33" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36287 } } peers { permanent_uuid: "17c92e258d644f448c9684ec7fc3d524" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39157 } } peers { permanent_uuid: "e24373b270894e3bb6e91e8550465a46" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45045 } }
I20260809 19:26:24.965731 699 raft_consensus.cc:374] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:24.965781 699 raft_consensus.cc:729] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e24373b270894e3bb6e91e8550465a46, State: Initialized, Role: FOLLOWER
I20260809 19:26:24.965896 699 consensus_queue.cc:260] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5499a666793446f0a25ab69684366c33" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36287 } } peers { permanent_uuid: "17c92e258d644f448c9684ec7fc3d524" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39157 } } peers { permanent_uuid: "e24373b270894e3bb6e91e8550465a46" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45045 } }
I20260809 19:26:24.965931 700 raft_consensus.cc:348] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5499a666793446f0a25ab69684366c33" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36287 } } peers { permanent_uuid: "17c92e258d644f448c9684ec7fc3d524" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39157 } } peers { permanent_uuid: "e24373b270894e3bb6e91e8550465a46" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45045 } }
I20260809 19:26:24.965903 698 raft_consensus.cc:348] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5499a666793446f0a25ab69684366c33" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36287 } } peers { permanent_uuid: "17c92e258d644f448c9684ec7fc3d524" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39157 } } peers { permanent_uuid: "e24373b270894e3bb6e91e8550465a46" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45045 } }
I20260809 19:26:24.966058 700 raft_consensus.cc:374] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:24.966058 698 raft_consensus.cc:374] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:26:24.966091 700 raft_consensus.cc:729] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5499a666793446f0a25ab69684366c33, State: Initialized, Role: FOLLOWER
I20260809 19:26:24.966091 698 raft_consensus.cc:729] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17c92e258d644f448c9684ec7fc3d524, State: Initialized, Role: FOLLOWER
I20260809 19:26:24.966209 699 ts_tablet_manager.cc:1428] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:26:24.966230 700 consensus_queue.cc:260] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5499a666793446f0a25ab69684366c33" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36287 } } peers { permanent_uuid: "17c92e258d644f448c9684ec7fc3d524" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39157 } } peers { permanent_uuid: "e24373b270894e3bb6e91e8550465a46" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45045 } }
I20260809 19:26:24.966218 698 consensus_queue.cc:260] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5499a666793446f0a25ab69684366c33" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36287 } } peers { permanent_uuid: "17c92e258d644f448c9684ec7fc3d524" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39157 } } peers { permanent_uuid: "e24373b270894e3bb6e91e8550465a46" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45045 } }
I20260809 19:26:24.966331 543 heartbeater.cc:503] Master 127.21.81.254:40007 was elected leader, sending a full tablet report...
I20260809 19:26:24.966501 700 ts_tablet_manager.cc:1428] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:26:24.966501 698 ts_tablet_manager.cc:1428] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 19:26:24.966519 680 heartbeater.cc:503] Master 127.21.81.254:40007 was elected leader, sending a full tablet report...
I20260809 19:26:24.966611 405 heartbeater.cc:503] Master 127.21.81.254:40007 was elected leader, sending a full tablet report...
I20260809 19:26:24.975473 705 raft_consensus.cc:482] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:26:24.975586 705 raft_consensus.cc:504] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5499a666793446f0a25ab69684366c33" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36287 } } peers { permanent_uuid: "17c92e258d644f448c9684ec7fc3d524" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39157 } } peers { permanent_uuid: "e24373b270894e3bb6e91e8550465a46" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45045 } }
I20260809 19:26:24.975950 705 leader_election.cc:288] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5499a666793446f0a25ab69684366c33 (127.21.81.193:36287), e24373b270894e3bb6e91e8550465a46 (127.21.81.194:45045)
I20260809 19:26:24.978844 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.976107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:45045 (local address 127.21.81.195:49613)
0809 19:26:24.976171 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:24.976182 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:24.976225 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:24.976376 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:24.976376 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:24.976378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:26:24.976452 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.976453 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.977128 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.977129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.978025 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.978027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.978578 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.978579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.978664 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.978668 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:26:24.978725 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 19:26:24.978756 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":46}
I20260809 19:26:24.978852 696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.976183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:49613 (local address 127.21.81.194:45045)
0809 19:26:24.976234 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:24.976236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:24.976249 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:26:24.976293 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:24.976294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:24.976307 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:24.976374 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:26:24.976538 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.977054 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.978108 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.978505 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.978561 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.978568 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:24.978741 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 19:26:24.978788 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:26:24.978914 693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.976274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:48801 (local address 127.21.81.193:36287)
0809 19:26:24.976311 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:24.976313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:24.976361 (+ 48us) server_negotiation.cc:407] Connection header received
0809 19:26:24.976419 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:24.976420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:24.976432 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:24.976488 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:26:24.976681 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.977137 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.978137 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.978570 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.978653 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.978661 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:24.978812 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 19:26:24.978847 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:26:24.978973 707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.976122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:36287 (local address 127.21.81.195:48801)
0809 19:26:24.976296 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:24.976306 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:26:24.976372 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:26:24.976536 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:26:24.976537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:26:24.976538 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:26:24.976611 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.976613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.977221 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.977221 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.978036 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:26:24.978038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.978656 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.978657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:26:24.978735 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.978739 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:26:24.978794 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 19:26:24.978830 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":90,"threads_started":1}
I20260809 19:26:24.979192 496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0ba02f33ad141caaa3eeab5aa4402cc" candidate_uuid: "17c92e258d644f448c9684ec7fc3d524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e24373b270894e3bb6e91e8550465a46" is_pre_election: true
I20260809 19:26:24.979331 496 raft_consensus.cc:2454] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17c92e258d644f448c9684ec7fc3d524 in term 0.
I20260809 19:26:24.979463 355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0ba02f33ad141caaa3eeab5aa4402cc" candidate_uuid: "17c92e258d644f448c9684ec7fc3d524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5499a666793446f0a25ab69684366c33" is_pre_election: true
I20260809 19:26:24.979571 565 leader_election.cc:302] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17c92e258d644f448c9684ec7fc3d524, e24373b270894e3bb6e91e8550465a46; no voters:
I20260809 19:26:24.979604 355 raft_consensus.cc:2454] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17c92e258d644f448c9684ec7fc3d524 in term 0.
I20260809 19:26:24.979678 705 raft_consensus.cc:2781] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:26:24.979734 705 raft_consensus.cc:482] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:26:24.979761 705 raft_consensus.cc:3037] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:24.980548 705 raft_consensus.cc:504] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5499a666793446f0a25ab69684366c33" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36287 } } peers { permanent_uuid: "17c92e258d644f448c9684ec7fc3d524" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39157 } } peers { permanent_uuid: "e24373b270894e3bb6e91e8550465a46" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45045 } }
I20260809 19:26:24.980685 705 leader_election.cc:288] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [CANDIDATE]: Term 1 election: Requested vote from peers 5499a666793446f0a25ab69684366c33 (127.21.81.193:36287), e24373b270894e3bb6e91e8550465a46 (127.21.81.194:45045)
I20260809 19:26:24.980855 496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0ba02f33ad141caaa3eeab5aa4402cc" candidate_uuid: "17c92e258d644f448c9684ec7fc3d524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e24373b270894e3bb6e91e8550465a46"
I20260809 19:26:24.980871 355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0ba02f33ad141caaa3eeab5aa4402cc" candidate_uuid: "17c92e258d644f448c9684ec7fc3d524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5499a666793446f0a25ab69684366c33"
I20260809 19:26:24.980931 496 raft_consensus.cc:3037] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:24.980931 355 raft_consensus.cc:3037] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:26:24.981841 355 raft_consensus.cc:2454] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17c92e258d644f448c9684ec7fc3d524 in term 1.
I20260809 19:26:24.981841 496 raft_consensus.cc:2454] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17c92e258d644f448c9684ec7fc3d524 in term 1.
I20260809 19:26:24.982015 567 leader_election.cc:302] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17c92e258d644f448c9684ec7fc3d524, 5499a666793446f0a25ab69684366c33; no voters:
I20260809 19:26:24.982146 705 raft_consensus.cc:2781] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:26:24.982290 705 raft_consensus.cc:686] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [term 1 LEADER]: Becoming Leader. State: Replica: 17c92e258d644f448c9684ec7fc3d524, State: Running, Role: LEADER
I20260809 19:26:24.982388 705 consensus_queue.cc:237] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5499a666793446f0a25ab69684366c33" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36287 } } peers { permanent_uuid: "17c92e258d644f448c9684ec7fc3d524" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39157 } } peers { permanent_uuid: "e24373b270894e3bb6e91e8550465a46" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45045 } }
I20260809 19:26:24.983278 32672 catalog_manager.cc:5495] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17c92e258d644f448c9684ec7fc3d524 (127.21.81.195). New cstate: current_term: 1 leader_uuid: "17c92e258d644f448c9684ec7fc3d524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5499a666793446f0a25ab69684366c33" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17c92e258d644f448c9684ec7fc3d524" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39157 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e24373b270894e3bb6e91e8550465a46" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45045 } health_report { overall_health: UNKNOWN } } }
I20260809 19:26:24.992036 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:24.989684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57082 (local address 127.21.81.195:39157)
0809 19:26:24.989744 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:26:24.989746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:26:24.989756 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:26:24.989797 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:26:24.989798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:26:24.989813 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:26:24.989893 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:26:24.990029 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.990478 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.991250 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:26:24.991452 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:26:24.991534 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:26:24.991707 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:26:24.991803 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:26:24.991895 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 19:26:24.991923 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 19:26:24.991942 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:26:24.993637 355 raft_consensus.cc:1261] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33 [term 1 FOLLOWER]: Refusing update from remote peer 17c92e258d644f448c9684ec7fc3d524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:26:24.993656 496 raft_consensus.cc:1261] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46 [term 1 FOLLOWER]: Refusing update from remote peer 17c92e258d644f448c9684ec7fc3d524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:26:24.993952 705 consensus_queue.cc:1035] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5499a666793446f0a25ab69684366c33" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:26:24.994064 708 consensus_queue.cc:1035] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e24373b270894e3bb6e91e8550465a46" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:26:24.996270 711 mvcc.cc:205] Tried to move back new op lower bound from 7316699484131610624 to 7316699484088594432. Current Snapshot: MvccSnapshot[applied={T|T < 7316699484131610624}]
I20260809 19:26:24.996727 713 mvcc.cc:205] Tried to move back new op lower bound from 7316699484131610624 to 7316699484088594432. Current Snapshot: MvccSnapshot[applied={T|T < 7316699484131610624}]
I20260809 19:26:24.996918 712 mvcc.cc:205] Tried to move back new op lower bound from 7316699484131610624 to 7316699484088594432. Current Snapshot: MvccSnapshot[applied={T|T < 7316699484131610624}]
I20260809 19:26:24.998416 32672 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:26:24.998524 32672 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:26:24.999190 32672 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05eb8c7a162f462b9b0302be6ed0b383: Sending DeleteTablet for 3 replicas of tablet e0ba02f33ad141caaa3eeab5aa4402cc
I20260809 19:26:24.999508 335 tablet_service.cc:1514] Processing DeleteTablet for tablet e0ba02f33ad141caaa3eeab5aa4402cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:24 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:42718
I20260809 19:26:24.999521 476 tablet_service.cc:1514] Processing DeleteTablet for tablet e0ba02f33ad141caaa3eeab5aa4402cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:24 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:37050
I20260809 19:26:24.999807 722 tablet_replica.cc:330] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33: stopping tablet replica
I20260809 19:26:24.999807 613 tablet_service.cc:1514] Processing DeleteTablet for tablet e0ba02f33ad141caaa3eeab5aa4402cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:26:24 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:57078
I20260809 19:26:24.999933 722 raft_consensus.cc:2229] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:26:25.000041 722 raft_consensus.cc:2258] T e0ba02f33ad141caaa3eeab5aa4402cc P 5499a666793446f0a25ab69684366c33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:25.000048 724 tablet_replica.cc:330] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524: stopping tablet replica
I20260809 19:26:25.000283 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 32737
I20260809 19:26:25.008248 723 tablet_replica.cc:330] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46: stopping tablet replica
I20260809 19:26:25.008368 723 raft_consensus.cc:2229] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:26:25.008477 723 raft_consensus.cc:2258] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:25.008960 724 raft_consensus.cc:2229] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:26:25.009068 724 raft_consensus.cc:2258] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:26:25.009977 724 ts_tablet_manager.cc:1905] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:26:25.011962 723 ts_tablet_manager.cc:1905] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:26:25.011978 724 ts_tablet_manager.cc:1918] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:26:25.012171 724 log.cc:1196] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/e0ba02f33ad141caaa3eeab5aa4402cc
I20260809 19:26:25.012540 724 ts_tablet_manager.cc:1939] T e0ba02f33ad141caaa3eeab5aa4402cc P 17c92e258d644f448c9684ec7fc3d524: Deleting consensus metadata
I20260809 19:26:25.013597 32660 catalog_manager.cc:4841] TS 17c92e258d644f448c9684ec7fc3d524 (127.21.81.195:39157): tablet e0ba02f33ad141caaa3eeab5aa4402cc (table test-table [id=d795f3295144486a9c88680941397141]) successfully deleted
I20260809 19:26:25.013638 723 ts_tablet_manager.cc:1918] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:26:25.013715 723 log.cc:1196] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/e0ba02f33ad141caaa3eeab5aa4402cc
I20260809 19:26:25.013980 723 ts_tablet_manager.cc:1939] T e0ba02f33ad141caaa3eeab5aa4402cc P e24373b270894e3bb6e91e8550465a46: Deleting consensus metadata
W20260809 19:26:25.014046 32659 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:36287 (error 108)
I20260809 19:26:25.014379 32657 catalog_manager.cc:4841] TS e24373b270894e3bb6e91e8550465a46 (127.21.81.194:45045): tablet e0ba02f33ad141caaa3eeab5aa4402cc (table test-table [id=d795f3295144486a9c88680941397141]) successfully deleted
I20260809 19:26:25.014593 697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:26:25.014460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:36287 (local address 127.0.0.1:42728)
0809 19:26:25.014509 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 19:26:25.014544 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:36287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 19:26:25.014810 32659 catalog_manager.cc:4584] TS 5499a666793446f0a25ab69684366c33 (127.21.81.193:36287): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e0ba02f33ad141caaa3eeab5aa4402cc: Network error: Client connection negotiation failed: client connection to 127.21.81.193:36287: connect: Connection refused (error 111)
I20260809 19:26:25.015019 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 410
I20260809 19:26:25.022028 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 548
I20260809 19:26:25.028730 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 32640
2026-08-09T19:26:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1952 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:26:25 dist-test-slave-njxd krb5kdc[728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:26:25 dist-test-slave-njxd krb5kdc[728](info): set up 2 sockets
Aug 09 19:26:25 dist-test-slave-njxd krb5kdc[728](info): commencing operation
krb5kdc: starting...
W20260809 19:26:26.118588 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:26:26 dist-test-slave-njxd krb5kdc[728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:26:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:26:26Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.81.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.81.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/rangeradmin_127.21.81.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/rangeradmin_127.21.81.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.81.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.81.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/rangerlookup_127.21.81.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/rangerlookup_127.21.81.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.81.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.81.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/HTTP_127.21.81.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/HTTP_127.21.81.212@KRBTEST.COM.keytab.
I20260809 19:26:26.282637 21831 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskkvvQ95/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/postgres -l logfile start
2026-08-09 19:26:28.781 UTC [759] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 19:26:28.781 UTC [759] LOG: listening on IPv4 address "127.21.81.212", port 54085
2026-08-09 19:26:28.785 UTC [759] LOG: listening on Unix socket "/tmp/.s.PGSQL.54085"
2026-08-09 19:26:28.821 UTC [762] LOG: database system was shut down at 2026-08-09 19:26:27 UTC
2026-08-09 19:26:28.825 UTC [759] LOG: database system is ready to accept connections
I20260809 19:26:29.806300 21831 mini_postgres.cc:96] Postgres bound to 54085
2026-08-09 19:26:29.815 UTC [770] FATAL: database "slave" does not exist
127.21.81.212:54085 - accepting connections
I20260809 19:26:29.816417 21831 mini_ranger.cc:161] Starting Ranger...
I20260809 19:26:29.833539 21831 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 19:26:30.280865 21831 mini_ranger.cc:87] Created ranger Postgres database
I20260809 19:26:30.280961 21831 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-admin
2026-08-09 19:26:30,664 [I] DB FLAVOR :POSTGRES
2026-08-09 19:26:30,665 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 19:26:30,665 [I] Checking connection..
2026-08-09 19:26:30,665 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 19:26:30,928 [I] Checking connection passed.
2026-08-09 19:26:30,928 [I] --------- Verifying version history table ---------
2026-08-09 19:26:30,928 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:26:31,192 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 19:26:31,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:26:31,460 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 19:26:31,460 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 19:26:31,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 19:26:31.724 UTC [837] WARNING: there is no transaction in progress
2026-08-09 19:26:31,732 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 19:26:31,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:26:31,979 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 19:26:31,980 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 19:26:31,980 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 19:26:32,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 19:26:32,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-njxd.c.gcp-upstream.internal','N') ;"
2026-08-09 19:26:32,753 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 19:26:33,003 [I] Table x_portal_user does not exist in database ranger
2026-08-09 19:26:33,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 19:26:33,257 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 19:26:33,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 19:26:33,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 19:26:33,789 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 19:26:33,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 19:26:34.720 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:34.737 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:34.753 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:34.769 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:34.784 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:34.805 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:34.861 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:34.867 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:34.881 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:34.891 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:34.901 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:34.915 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:34.925 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:34.932 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:34.939 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:34.945 UTC [946] WARNING: there is no transaction in progress
2026-08-09 19:26:35,561 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 19:26:35,562 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 19:26:35,826 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 19:26:35,826 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 19:26:36,074 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 19:26:36,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 19:26:36,347 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 19:26:36,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 19:26:36,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 19:26:36,865 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-njxd.c.gcp-upstream.internal';"
2026-08-09 19:26:37,119 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 19:26:37,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 19:26:37,363 [I] DB_PATCHES have already been applied
I20260809 19:26:37.369326 21831 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-admin:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskkvvQ95/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 19:26:37.369403 21831 mini_ranger.cc:193] Using host: 127.21.81.212
I20260809 19:26:37.373339 21831 mini_ranger.cc:227] Ranger admin URL: http://127.21.81.212:41679
Aug 09, 2026 7:26:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:26:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:26:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 7:26:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 7:26:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 7:26:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 7:26:38 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 19:26:39.671952 21831 mini_ranger.cc:160] Time spent starting Ranger: real 9.856s user 0.000s sys 0.014s
Aug 09, 2026 7:27:00 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 7:27:00 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 7:27:00 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 7:27:00 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 7:27:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 7:27:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 7:27:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 7:27:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 7:27:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 7:27:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 7:27:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 7:27:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 7:27:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 7:27:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 7:27:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 7:27:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 7:27:00 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 7:27:01 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 19:27:02.185273 21831 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.81.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.81.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/rangerkms_127.21.81.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/rangerkms_127.21.81.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.81.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.81.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.81.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/HTTP_127.21.81.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.81.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/HTTP_127.21.81.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 19:27:02 dist-test-slave-njxd krb5kdc[728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303622, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 19:27:02.292272 21831 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 19:27:02.305130 21831 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 19:27:03.129698 21831 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 19:27:03.129784 21831 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-kms
I20260809 19:27:03.129803 21831 mini_ranger_kms.cc:227] Using postgres at 127.21.81.212:54085
2026-08-09 19:27:03,602 [I] DB FLAVOR :POSTGRES
2026-08-09 19:27:03,603 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 19:27:03,603 [I] Checking connection
2026-08-09 19:27:03,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 19:27:03,888 [I] connection success
2026-08-09 19:27:03,888 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 19:27:03,888 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 19:27:03,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 19:27:04,166 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 19:27:04,166 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 19:27:04,166 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 19:27:04,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:54085/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 19:27:04,494 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 19:27:05.001089 21831 mini_ranger_kms.cc:310] Created kms service
I20260809 19:27:05.186873 21831 mini_ranger_kms.cc:326] Created kudu user
I20260809 19:27:05.236606 21831 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 19:27:05 dist-test-slave-njxd krb5kdc[728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303622, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
I20260809 19:27:05.695842 21831 mini_ranger_kms.cc:384] Added ranger policy
I20260809 19:27:05.695963 21831 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-kms:/tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 19:27:05.695989 21831 mini_ranger_kms.cc:240] Using host: 127.21.81.212
I20260809 19:27:05.700555 21831 mini_ranger_kms.cc:276] Ranger KMS PID: 1136
I20260809 19:27:05.700680 21831 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.81.212:46337
Aug 09, 2026 7:27:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:27:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:27:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 7:27:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 7:27:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 7:27:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 19:27:08.057549 21831 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.765s user 0.000s sys 0.450s
I20260809 19:27:08.057721 21831 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 19:27:08.057811 21831 mini_ranger_kms.cc:401] 127.21.81.212:46337/kms/v1/keys
Aug 09 19:27:08 dist-test-slave-njxd krb5kdc[728](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786303628, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.81.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 19:27:08 dist-test-slave-njxd krb5kdc[728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303628, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.81.212@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
Aug 09 19:27:09 dist-test-slave-njxd krb5kdc[728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303628, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.81.212@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
Aug 09 19:27:11 dist-test-slave-njxd krb5kdc[728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:27:11.412598 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:43371
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:43371
--ranger_config_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.81.212:46337/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:27:11.510865 1189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:27:11.511082 1189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:27:11.511121 1189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:27:11.512231 1189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:27:11.512297 1189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:27:11.512321 1189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:27:11.512353 1189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:27:11.512377 1189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:27:11.515025 1189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.81.212:46337/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:43371
--ranger_config_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:43371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:27:11.515292 1189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:27:11.515528 1189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:27:11.520812 1195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:27:11.520812 1197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:27:11.520924 1189 server_base.cc:1034] running on GCE node
W20260809 19:27:11.520978 1194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:27:11.521426 1189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:27:11.521792 1189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:27:11.522961 1189 hybrid_clock.cc:648] HybridClock initialized: now 1786303631522951 us; error 28 us; skew 500 ppm
Aug 09 19:27:11 dist-test-slave-njxd krb5kdc[728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303631, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:27:11.528707 1189 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:27:11.529237 1189 webserver.cc:460] Webserver started at http://127.21.81.254:38829/ using document root <none> and password file <none>
I20260809 19:27:11.529488 1189 fs_manager.cc:359] Metadata directory not provided
I20260809 19:27:11.529547 1189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:27:11.529728 1189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:27:11 dist-test-slave-njxd krb5kdc[728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303631, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
I20260809 19:27:11.578964 1189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "3617040d4b754e6fa86687c3f548c957"
format_stamp: "Formatted at 2026-08-09 19:27:11 on dist-test-slave-njxd"
server_key: "b5661ae6bb3802d52cbe64393ccde55a"
server_key_iv: "2f1b3a993138d00dca5edfacf4154d77"
server_key_version: "kuduclusterkey@0"
I20260809 19:27:11.579391 1189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "3617040d4b754e6fa86687c3f548c957"
format_stamp: "Formatted at 2026-08-09 19:27:11 on dist-test-slave-njxd"
server_key: "b5661ae6bb3802d52cbe64393ccde55a"
server_key_iv: "2f1b3a993138d00dca5edfacf4154d77"
server_key_version: "kuduclusterkey@0"
I20260809 19:27:11.580847 1189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:27:11.593667 1204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:27:11.593926 1189 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:27:11.594036 1189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "3617040d4b754e6fa86687c3f548c957"
format_stamp: "Formatted at 2026-08-09 19:27:11 on dist-test-slave-njxd"
server_key: "b5661ae6bb3802d52cbe64393ccde55a"
server_key_iv: "2f1b3a993138d00dca5edfacf4154d77"
server_key_version: "kuduclusterkey@0"
I20260809 19:27:11.594149 1189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:27:11.613957 1189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:27:11.615854 1189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:27:11.616081 1189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:27:11.622480 1189 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:43371
I20260809 19:27:11.622509 1260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:43371 every 8 connection(s)
I20260809 19:27:11.622890 1189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:27:11.624267 1261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:27:11.626916 1261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957: Bootstrap starting.
I20260809 19:27:11.627687 1261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957: Neither blocks nor log segments found. Creating new log.
I20260809 19:27:11.628114 1261 log.cc:824] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957: Log is configured to *not* fsync() on all Append() calls
I20260809 19:27:11.628804 1261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957: No bootstrap required, opened a new log
I20260809 19:27:11.629168 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 1189
I20260809 19:27:11.629317 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:27:11.629513 21831 external_mini_cluster.cc:1428] Setting key 9f4c30cc911228ff06944e1316e7cf70
I20260809 19:27:11.631820 1261 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3617040d4b754e6fa86687c3f548c957" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43371 } }
I20260809 19:27:11.631989 1261 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:27:11.632050 1261 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3617040d4b754e6fa86687c3f548c957, State: Initialized, Role: FOLLOWER
I20260809 19:27:11.632233 1261 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3617040d4b754e6fa86687c3f548c957" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43371 } }
I20260809 19:27:11.632328 1261 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:27:11.632361 1261 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:27:11.632439 1261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:27:11.633287 1261 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3617040d4b754e6fa86687c3f548c957" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43371 } }
I20260809 19:27:11.633414 1261 leader_election.cc:302] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3617040d4b754e6fa86687c3f548c957; no voters:
I20260809 19:27:11.633626 1261 leader_election.cc:288] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:27:11.633668 1266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:27:11.633927 1266 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [term 1 LEADER]: Becoming Leader. State: Replica: 3617040d4b754e6fa86687c3f548c957, State: Running, Role: LEADER
I20260809 19:27:11.633996 1261 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:27:11.634157 1266 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3617040d4b754e6fa86687c3f548c957" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43371 } }
I20260809 19:27:11.635788 1266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3617040d4b754e6fa86687c3f548c957. Latest consensus state: current_term: 1 leader_uuid: "3617040d4b754e6fa86687c3f548c957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3617040d4b754e6fa86687c3f548c957" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43371 } } }
I20260809 19:27:11.635923 1266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [sys.catalog]: This master's current role is: LEADER
I20260809 19:27:11.636144 1282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:27:11.636247 1261 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 19:27:11.636729 1282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:27:11.636755 1267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3617040d4b754e6fa86687c3f548c957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3617040d4b754e6fa86687c3f548c957" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43371 } } }
I20260809 19:27:11.636829 1267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957 [sys.catalog]: This master's current role is: LEADER
Aug 09 19:27:11 dist-test-slave-njxd krb5kdc[728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:27:11.644112 1285 mvcc.cc:205] Tried to move back new op lower bound from 7316699675188420608 to 7316699675174293504. Current Snapshot: MvccSnapshot[applied={T|T < 7316699675188420608}]
I20260809 19:27:11.646294 1282 catalog_manager.cc:1347] Generated new cluster ID: 121212d890074cf1ba804148151d0e90
I20260809 19:27:11.646356 1282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:27:11.651465 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:11.630217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41542 (local address 127.21.81.254:43371)
0809 19:27:11.630425 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 19:27:11.630435 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:27:11.630466 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:27:11.631832 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:27:11.631842 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:27:11.631924 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:27:11.632069 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:27:11.632981 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:11.634486 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:11.635707 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:11.637478 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:11.637646 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:11.641421 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:27:11.641523 (+ 102us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:27:11.641523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:27:11.641651 (+ 128us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:27:11.645407 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:27:11.645945 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:27:11.645949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:27:11.645950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:27:11.646221 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:27:11.649888 (+ 3667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:27:11.649889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:27:11.649889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:27:11.650368 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:27:11.650546 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 19:27:11.651042 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 19:27:11.651220 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
I20260809 19:27:11.682224 1282 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:27:11.683022 1282 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:27:11.689378 1282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957: Generated new TSK 0
I20260809 19:27:11.689566 1282 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:27:12.500406 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.81.212:46337/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43371
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:27:12.629729 1309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:27:12.629947 1309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:27:12.629989 1309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:27:12.631181 1309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:27:12.631245 1309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:27:12.631289 1309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:27:12.632793 1309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.81.212:46337/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:27:12.633044 1309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:27:12.633286 1309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:27:12.639901 1316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:27:12.640369 1309 server_base.cc:1034] running on GCE node
W20260809 19:27:12.640483 1315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:27:12.640301 1318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:27:12.640774 1309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:27:12.641044 1309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:27:12.642206 1309 hybrid_clock.cc:648] HybridClock initialized: now 1786303632642186 us; error 32 us; skew 500 ppm
Aug 09 19:27:12 dist-test-slave-njxd krb5kdc[728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303632, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:27:12.645506 1309 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:27:12.646059 1309 webserver.cc:460] Webserver started at http://127.21.81.193:39515/ using document root <none> and password file <none>
I20260809 19:27:12.646337 1309 fs_manager.cc:359] Metadata directory not provided
I20260809 19:27:12.646407 1309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:27:12.646543 1309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:27:12 dist-test-slave-njxd krb5kdc[728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303632, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
I20260809 19:27:12.662379 1309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "2006df36d636448ca48345eae800cdde"
format_stamp: "Formatted at 2026-08-09 19:27:12 on dist-test-slave-njxd"
server_key: "495ffe20576cd74590494c32356f4bc9"
server_key_iv: "e9fc8dec6dba26c7d8e47b02d5c1ac33"
server_key_version: "kuduclusterkey@0"
I20260809 19:27:12.662906 1309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "2006df36d636448ca48345eae800cdde"
format_stamp: "Formatted at 2026-08-09 19:27:12 on dist-test-slave-njxd"
server_key: "495ffe20576cd74590494c32356f4bc9"
server_key_iv: "e9fc8dec6dba26c7d8e47b02d5c1ac33"
server_key_version: "kuduclusterkey@0"
I20260809 19:27:12.664979 1309 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 19:27:12.680294 1328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:27:12.680482 1309 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:27:12.680589 1309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "2006df36d636448ca48345eae800cdde"
format_stamp: "Formatted at 2026-08-09 19:27:12 on dist-test-slave-njxd"
server_key: "495ffe20576cd74590494c32356f4bc9"
server_key_iv: "e9fc8dec6dba26c7d8e47b02d5c1ac33"
server_key_version: "kuduclusterkey@0"
I20260809 19:27:12.680698 1309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:27:12.695267 1309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:27:12.698474 1309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:27:12.698719 1309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:27:12.699146 1309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:27:12.699537 1309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:27:12.699584 1309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:27:12.699611 1309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:27:12.699636 1309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:27:12.708771 1309 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:46527
I20260809 19:27:12.709165 1447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:46527 every 8 connection(s)
I20260809 19:27:12.709313 1309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:27:12 dist-test-slave-njxd krb5kdc[728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303632, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:27:12.717829 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 1309
I20260809 19:27:12.717950 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:27:12.718147 21831 external_mini_cluster.cc:1428] Setting key 6375d40a7d46fd6fba6366181f4561e3
I20260809 19:27:12.721271 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:12.710241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:59589 (local address 127.21.81.254:43371)
0809 19:27:12.710408 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 19:27:12.710411 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:27:12.710700 (+ 289us) server_negotiation.cc:407] Connection header received
0809 19:27:12.711458 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:27:12.711460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:27:12.711475 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:27:12.711560 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:27:12.712354 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:12.713159 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:12.714790 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:12.715107 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:12.715185 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:12.717872 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:27:12.717874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:27:12.717874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:27:12.717894 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:27:12.719654 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:27:12.720169 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:27:12.720170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:27:12.720171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:27:12.720240 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:27:12.720532 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:27:12.720533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:27:12.720533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:27:12.720660 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:27:12.720704 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:27:12.721072 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 19:27:12.721172 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":74,"threads_started":1}
I20260809 19:27:12.721832 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:12.710246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:43371 (local address 127.21.81.193:59589)
0809 19:27:12.710515 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 19:27:12.710563 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 19:27:12.711325 (+ 762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:27:12.711664 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:27:12.711668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:27:12.711910 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:27:12.712229 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:12.712235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:12.713283 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:12.713285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:12.714675 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:12.714678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:12.715220 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:12.715221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:12.715370 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:12.716005 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:27:12.716015 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:27:12.717760 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:27:12.719746 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:27:12.719752 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:27:12.719754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:27:12.720055 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:27:12.720316 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:27:12.720316 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:27:12.720317 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:27:12.720452 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:27:12.720750 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:27:12.720757 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:27:12.720912 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 19:27:12.721412 (+ 500us) client_negotiation.cc:240] Negotiation successful
0809 19:27:12.721608 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":99,"threads_started":1}
I20260809 19:27:12.722297 1448 heartbeater.cc:347] Connected to a master server at 127.21.81.254:43371
I20260809 19:27:12.722440 1448 heartbeater.cc:464] Registering TS with master...
I20260809 19:27:12.722683 1448 heartbeater.cc:511] Master 127.21.81.254:43371 requested a full tablet report, sending...
I20260809 19:27:12.723312 1223 ts_manager.cc:194] Registered new tserver with Master: 2006df36d636448ca48345eae800cdde (127.21.81.193:46527)
I20260809 19:27:12.724258 1223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:59589
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:27:12.775840 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.81.212:46337/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43371
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:27:12 dist-test-slave-njxd krb5kdc[728](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786303632, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 19:27:12.913702 1456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:27:12.913981 1456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:27:12.914016 1456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:27:12.915699 1456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:27:12.915755 1456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:27:12.915791 1456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:27:12.918300 1456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.81.212:46337/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:27:12.918537 1456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:27:12.918802 1456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:27:12.926180 1462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:27:12.926769 1461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:27:12.926970 1464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:27:12.927037 1456 server_base.cc:1034] running on GCE node
I20260809 19:27:12.927314 1456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:27:12.927656 1456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:27:12.928825 1456 hybrid_clock.cc:648] HybridClock initialized: now 1786303632928788 us; error 48 us; skew 500 ppm
Aug 09 19:27:12 dist-test-slave-njxd krb5kdc[728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303632, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:27:12.932211 1456 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:27:12.932677 1456 webserver.cc:460] Webserver started at http://127.21.81.194:45657/ using document root <none> and password file <none>
I20260809 19:27:12.932897 1456 fs_manager.cc:359] Metadata directory not provided
I20260809 19:27:12.932941 1456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:27:12.933054 1456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:27:12 dist-test-slave-njxd krb5kdc[728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303632, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
I20260809 19:27:12.949528 1456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "c1c6c2a3206d4828ad23fcd650515221"
format_stamp: "Formatted at 2026-08-09 19:27:12 on dist-test-slave-njxd"
server_key: "dca9de6ad6269661ed5e67d09cc99223"
server_key_iv: "d3e71154421669e258ac787940e34e22"
server_key_version: "kuduclusterkey@0"
I20260809 19:27:12.950073 1456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "c1c6c2a3206d4828ad23fcd650515221"
format_stamp: "Formatted at 2026-08-09 19:27:12 on dist-test-slave-njxd"
server_key: "dca9de6ad6269661ed5e67d09cc99223"
server_key_iv: "d3e71154421669e258ac787940e34e22"
server_key_version: "kuduclusterkey@0"
I20260809 19:27:12.954541 1456 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260809 19:27:12.968734 1472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:27:12.968904 1456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:27:12.968999 1456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "c1c6c2a3206d4828ad23fcd650515221"
format_stamp: "Formatted at 2026-08-09 19:27:12 on dist-test-slave-njxd"
server_key: "dca9de6ad6269661ed5e67d09cc99223"
server_key_iv: "d3e71154421669e258ac787940e34e22"
server_key_version: "kuduclusterkey@0"
I20260809 19:27:12.969097 1456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:27:12.989713 1456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:27:12.992168 1456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:27:12.992307 1456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:27:12.992509 1456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:27:12.992887 1456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:27:12.992934 1456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:27:12.992970 1456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:27:12.992987 1456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:27:13.008183 1456 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:38153
I20260809 19:27:13.008782 1456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:27:13.009877 1591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:38153 every 8 connection(s)
I20260809 19:27:13.013880 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 1456
I20260809 19:27:13.014010 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:27:13.014215 21831 external_mini_cluster.cc:1428] Setting key f683f440fc0cbc4bc7744dfab6e3b809
Aug 09 19:27:13 dist-test-slave-njxd krb5kdc[728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303632, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:27:13.020318 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.010483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:51773 (local address 127.21.81.254:43371)
0809 19:27:13.010528 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:27:13.010530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:27:13.010542 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:27:13.011137 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:27:13.011139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:27:13.011155 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:27:13.011186 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:27:13.011975 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.012716 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.014172 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.014465 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.014505 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.017058 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:27:13.017060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:27:13.017060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:27:13.017068 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:27:13.018863 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:27:13.019309 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:27:13.019310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:27:13.019310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:27:13.019367 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:27:13.019630 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:27:13.019631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:27:13.019631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:27:13.019754 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:27:13.019794 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 19:27:13.020143 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 19:27:13.020231 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:27:13.020844 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.010151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:43371 (local address 127.21.81.194:51773)
0809 19:27:13.010378 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 19:27:13.010426 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 19:27:13.011008 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:27:13.011358 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:27:13.011362 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:27:13.011605 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:27:13.011867 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:13.011872 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.012831 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.012832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:13.014070 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:13.014073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.014627 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.014628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:13.014762 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.015349 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:27:13.015359 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:27:13.016951 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:27:13.018952 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:27:13.018958 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:27:13.018960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:27:13.019223 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:27:13.019430 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:27:13.019430 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:27:13.019431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:27:13.019550 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:27:13.019839 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:27:13.019845 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:27:13.019998 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 19:27:13.020456 (+ 458us) client_negotiation.cc:240] Negotiation successful
0809 19:27:13.020621 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":127,"threads_started":1}
I20260809 19:27:13.021293 1592 heartbeater.cc:347] Connected to a master server at 127.21.81.254:43371
I20260809 19:27:13.021378 1592 heartbeater.cc:464] Registering TS with master...
I20260809 19:27:13.021560 1592 heartbeater.cc:511] Master 127.21.81.254:43371 requested a full tablet report, sending...
I20260809 19:27:13.021917 1223 ts_manager.cc:194] Registered new tserver with Master: c1c6c2a3206d4828ad23fcd650515221 (127.21.81.194:38153)
I20260809 19:27:13.022384 1223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:51773
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:27:13.049149 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.81.212:46337/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43371
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:27:13.191536 1598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:27:13.191908 1598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:27:13.191998 1598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:27:13.193913 1598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:27:13.194043 1598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:27:13.194139 1598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:27:13.196727 1598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.81.212:46337/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:43371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:27:13.197223 1598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:27:13.197546 1598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:27:13.204689 1604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:27:13.205140 1603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:27:13.205392 1606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:27:13.206614 1598 server_base.cc:1034] running on GCE node
I20260809 19:27:13.206935 1598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:27:13.207216 1598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:27:13.208388 1598 hybrid_clock.cc:648] HybridClock initialized: now 1786303633208358 us; error 40 us; skew 500 ppm
Aug 09 19:27:13 dist-test-slave-njxd krb5kdc[728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303633, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:27:13.212019 1598 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:27:13.212579 1598 webserver.cc:460] Webserver started at http://127.21.81.195:44743/ using document root <none> and password file <none>
I20260809 19:27:13.212844 1598 fs_manager.cc:359] Metadata directory not provided
I20260809 19:27:13.212894 1598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:27:13.213018 1598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:27:13 dist-test-slave-njxd krb5kdc[728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303633, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
I20260809 19:27:13.228209 1598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "f86102b7988a48799264c600d0e7be60"
format_stamp: "Formatted at 2026-08-09 19:27:13 on dist-test-slave-njxd"
server_key: "dc7e2ddcac70897dcdddbeb948430d91"
server_key_iv: "c846a85685fc724c2a47fa35407e72cb"
server_key_version: "kuduclusterkey@0"
I20260809 19:27:13.228807 1598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "f86102b7988a48799264c600d0e7be60"
format_stamp: "Formatted at 2026-08-09 19:27:13 on dist-test-slave-njxd"
server_key: "dc7e2ddcac70897dcdddbeb948430d91"
server_key_iv: "c846a85685fc724c2a47fa35407e72cb"
server_key_version: "kuduclusterkey@0"
I20260809 19:27:13.238251 1598 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.004s
I20260809 19:27:13.252934 1613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:27:13.253150 1598 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:27:13.253263 1598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "f86102b7988a48799264c600d0e7be60"
format_stamp: "Formatted at 2026-08-09 19:27:13 on dist-test-slave-njxd"
server_key: "dc7e2ddcac70897dcdddbeb948430d91"
server_key_iv: "c846a85685fc724c2a47fa35407e72cb"
server_key_version: "kuduclusterkey@0"
I20260809 19:27:13.253388 1598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:27:13.271323 1598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:27:13.273883 1598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:27:13.274037 1598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:27:13.274288 1598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:27:13.274729 1598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:27:13.274781 1598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:27:13.274819 1598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:27:13.274843 1598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:27:13.281823 1598 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:34155
I20260809 19:27:13.282281 1598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:27:13.287561 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 1598
I20260809 19:27:13.287667 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:27:13.287822 21831 external_mini_cluster.cc:1428] Setting key f65407f6865aa357e7f79493626927bb
I20260809 19:27:13.289034 1734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:34155 every 8 connection(s)
Aug 09 19:27:13 dist-test-slave-njxd krb5kdc[728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303633, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:27:13.311151 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.289980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:43485 (local address 127.21.81.254:43371)
0809 19:27:13.290024 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:27:13.290026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:27:13.292332 (+ 2306us) server_negotiation.cc:407] Connection header received
0809 19:27:13.293638 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:27:13.293639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:27:13.293651 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:27:13.293695 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:27:13.294788 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.295533 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.296985 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.297260 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.297298 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.300497 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:27:13.300498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:27:13.300499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:27:13.300506 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:27:13.302283 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:27:13.309642 (+ 7359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:27:13.309644 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:27:13.309644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:27:13.309731 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:27:13.310096 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:27:13.310097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:27:13.310097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:27:13.310254 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:27:13.310331 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:27:13.310945 (+ 614us) server_negotiation.cc:299] Negotiation successful
0809 19:27:13.311055 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:27:13.312101 1737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.289695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:43371 (local address 127.21.81.195:43485)
0809 19:27:13.292181 (+ 2486us) negotiation.cc:107] Waiting for socket to connect
0809 19:27:13.292234 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 19:27:13.293477 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:27:13.293890 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:27:13.293894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:27:13.294324 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:27:13.294666 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:13.294672 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.295645 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.295645 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:13.296885 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:13.296888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.297380 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.297380 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:13.297559 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.298296 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:27:13.298306 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:27:13.300394 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:27:13.305230 (+ 4836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:27:13.305240 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:27:13.305243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:27:13.305633 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:27:13.309823 (+ 4190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:27:13.309824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:27:13.309825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:27:13.310006 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:27:13.310463 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:27:13.310471 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:27:13.310673 (+ 202us) client_negotiation.cc:769] Sending connection context
0809 19:27:13.311583 (+ 910us) client_negotiation.cc:240] Negotiation successful
0809 19:27:13.311792 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2383,"thread_start_us":105,"threads_started":1}
I20260809 19:27:13.313915 1735 heartbeater.cc:347] Connected to a master server at 127.21.81.254:43371
I20260809 19:27:13.314028 1735 heartbeater.cc:464] Registering TS with master...
I20260809 19:27:13.314246 1735 heartbeater.cc:511] Master 127.21.81.254:43371 requested a full tablet report, sending...
I20260809 19:27:13.314640 1223 ts_manager.cc:194] Registered new tserver with Master: f86102b7988a48799264c600d0e7be60 (127.21.81.195:34155)
I20260809 19:27:13.315131 1223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:43485
I20260809 19:27:13.322110 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:27:13.338762 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.325468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41546 (local address 127.21.81.254:43371)
0809 19:27:13.325521 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:27:13.325523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:27:13.326015 (+ 492us) server_negotiation.cc:407] Connection header received
0809 19:27:13.326129 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:27:13.326131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:27:13.326142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:27:13.326175 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:27:13.326995 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.327774 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.329092 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.329379 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.329427 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.331140 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:27:13.331141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:27:13.331142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:27:13.331150 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:27:13.332915 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:27:13.337548 (+ 4633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:27:13.337549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:27:13.337550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:27:13.337663 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:27:13.338077 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:27:13.338078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:27:13.338078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:27:13.338221 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:27:13.338312 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 19:27:13.338561 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 19:27:13.338662 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:27:13.341356 1223 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:27:13.343876 1223 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:27:13.358748 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.349835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:38153 (local address 127.0.0.1:43684)
0809 19:27:13.353420 (+ 3585us) negotiation.cc:107] Waiting for socket to connect
0809 19:27:13.353430 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:27:13.353489 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:27:13.353780 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:27:13.353783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:27:13.353786 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:27:13.353863 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:13.353866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.355602 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.355603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:13.356843 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:13.356846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.358409 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.358410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:13.358492 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.358506 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 19:27:13.358590 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 19:27:13.358659 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3567,"thread_start_us":74,"threads_started":1}
I20260809 19:27:13.359215 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.350086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.21.81.194:38153)
0809 19:27:13.352208 (+ 2122us) server_negotiation.cc:206] Beginning negotiation
0809 19:27:13.352212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:27:13.353555 (+ 1343us) server_negotiation.cc:407] Connection header received
0809 19:27:13.353659 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:27:13.353662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:27:13.353679 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:27:13.353760 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:27:13.353951 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.355499 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.356965 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.358175 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.358249 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.358371 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 19:27:13.359091 (+ 720us) server_negotiation.cc:299] Negotiation successful
0809 19:27:13.359134 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2065,"thread_start_us":243,"threads_started":1}
I20260809 19:27:13.360306 1525 tablet_service.cc:1467] Processing CreateTablet for tablet 0592419e6e8c431ca58e30426f84c197 (DEFAULT_TABLE table=test-table [id=af824e15208f4ddab0414f759525507e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:27:13.360822 1525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0592419e6e8c431ca58e30426f84c197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:27:13.363850 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.348752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:34155 (local address 127.0.0.1:59736)
0809 19:27:13.350793 (+ 2041us) negotiation.cc:107] Waiting for socket to connect
0809 19:27:13.350836 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 19:27:13.350902 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:27:13.358020 (+ 7118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:27:13.358021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:27:13.358023 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:27:13.358071 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:13.358073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.359974 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.359975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:13.361073 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:13.361076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.363633 (+ 2557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.363634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:13.363700 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.363705 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:27:13.363736 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 19:27:13.363775 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1989,"thread_start_us":73,"threads_started":1}
I20260809 19:27:13.364295 1751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.349389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59736 (local address 127.21.81.195:34155)
0809 19:27:13.357718 (+ 8329us) server_negotiation.cc:206] Beginning negotiation
0809 19:27:13.357723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:27:13.357735 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:27:13.357814 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:27:13.357848 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:27:13.357867 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:27:13.358147 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:27:13.358333 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.359511 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.362637 (+ 3126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.363525 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.364039 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.364131 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 19:27:13.364158 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 19:27:13.364211 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8286,"thread_start_us":205,"threads_started":1}
I20260809 19:27:13.364950 1666 tablet_service.cc:1467] Processing CreateTablet for tablet 0592419e6e8c431ca58e30426f84c197 (DEFAULT_TABLE table=test-table [id=af824e15208f4ddab0414f759525507e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:27:13.365280 1666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0592419e6e8c431ca58e30426f84c197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:27:13.366658 1755 tablet_bootstrap.cc:492] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221: Bootstrap starting.
I20260809 19:27:13.368177 1755 tablet_bootstrap.cc:654] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221: Neither blocks nor log segments found. Creating new log.
I20260809 19:27:13.368762 1755 log.cc:824] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221: Log is configured to *not* fsync() on all Append() calls
I20260809 19:27:13.368887 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.349016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:46527 (local address 127.0.0.1:38006)
0809 19:27:13.351109 (+ 2093us) negotiation.cc:107] Waiting for socket to connect
0809 19:27:13.351118 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:27:13.351168 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:27:13.359834 (+ 8666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:27:13.359835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:27:13.359837 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:27:13.359876 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:13.359878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.364585 (+ 4707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.364586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:13.365616 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:13.365619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.368661 (+ 3042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.368662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:13.368733 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.368737 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:27:13.368768 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 19:27:13.368817 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2047,"thread_start_us":153,"threads_started":1}
I20260809 19:27:13.369380 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.349129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38006 (local address 127.21.81.193:46527)
0809 19:27:13.359662 (+ 10533us) server_negotiation.cc:206] Beginning negotiation
0809 19:27:13.359667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:27:13.359676 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:27:13.359714 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:27:13.359716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:27:13.359731 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:27:13.361209 (+ 1478us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:27:13.361284 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.362484 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.365800 (+ 3316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.366518 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.369058 (+ 2540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.369138 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 19:27:13.369165 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 19:27:13.369222 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10491,"thread_start_us":84,"threads_started":1}
I20260809 19:27:13.370142 1755 tablet_bootstrap.cc:492] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221: No bootstrap required, opened a new log
I20260809 19:27:13.370219 1755 ts_tablet_manager.cc:1397] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 19:27:13.370127 1379 tablet_service.cc:1467] Processing CreateTablet for tablet 0592419e6e8c431ca58e30426f84c197 (DEFAULT_TABLE table=test-table [id=af824e15208f4ddab0414f759525507e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:27:13.370476 1379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0592419e6e8c431ca58e30426f84c197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:27:13.372594 1755 raft_consensus.cc:348] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c6c2a3206d4828ad23fcd650515221" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38153 } } peers { permanent_uuid: "f86102b7988a48799264c600d0e7be60" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34155 } } peers { permanent_uuid: "2006df36d636448ca48345eae800cdde" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46527 } }
I20260809 19:27:13.372768 1755 raft_consensus.cc:374] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:27:13.372800 1755 raft_consensus.cc:729] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1c6c2a3206d4828ad23fcd650515221, State: Initialized, Role: FOLLOWER
I20260809 19:27:13.373535 1755 consensus_queue.cc:260] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c6c2a3206d4828ad23fcd650515221" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38153 } } peers { permanent_uuid: "f86102b7988a48799264c600d0e7be60" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34155 } } peers { permanent_uuid: "2006df36d636448ca48345eae800cdde" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46527 } }
I20260809 19:27:13.374143 1755 ts_tablet_manager.cc:1428] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 19:27:13.374315 1592 heartbeater.cc:503] Master 127.21.81.254:43371 was elected leader, sending a full tablet report...
I20260809 19:27:13.381786 1756 tablet_bootstrap.cc:492] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60: Bootstrap starting.
I20260809 19:27:13.383276 1756 tablet_bootstrap.cc:654] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60: Neither blocks nor log segments found. Creating new log.
I20260809 19:27:13.383800 1756 log.cc:824] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60: Log is configured to *not* fsync() on all Append() calls
I20260809 19:27:13.384608 1756 tablet_bootstrap.cc:492] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60: No bootstrap required, opened a new log
I20260809 19:27:13.384694 1756 ts_tablet_manager.cc:1397] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 19:27:13.385376 1758 tablet_bootstrap.cc:492] T 0592419e6e8c431ca58e30426f84c197 P 2006df36d636448ca48345eae800cdde: Bootstrap starting.
I20260809 19:27:13.386452 1758 tablet_bootstrap.cc:654] T 0592419e6e8c431ca58e30426f84c197 P 2006df36d636448ca48345eae800cdde: Neither blocks nor log segments found. Creating new log.
I20260809 19:27:13.386868 1758 log.cc:824] T 0592419e6e8c431ca58e30426f84c197 P 2006df36d636448ca48345eae800cdde: Log is configured to *not* fsync() on all Append() calls
I20260809 19:27:13.387506 1756 raft_consensus.cc:348] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c6c2a3206d4828ad23fcd650515221" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38153 } } peers { permanent_uuid: "f86102b7988a48799264c600d0e7be60" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34155 } } peers { permanent_uuid: "2006df36d636448ca48345eae800cdde" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46527 } }
I20260809 19:27:13.387702 1756 raft_consensus.cc:374] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:27:13.387735 1756 raft_consensus.cc:729] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f86102b7988a48799264c600d0e7be60, State: Initialized, Role: FOLLOWER
I20260809 19:27:13.387899 1756 consensus_queue.cc:260] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c6c2a3206d4828ad23fcd650515221" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38153 } } peers { permanent_uuid: "f86102b7988a48799264c600d0e7be60" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34155 } } peers { permanent_uuid: "2006df36d636448ca48345eae800cdde" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46527 } }
I20260809 19:27:13.388147 1756 ts_tablet_manager.cc:1428] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260809 19:27:13.388275 1735 heartbeater.cc:503] Master 127.21.81.254:43371 was elected leader, sending a full tablet report...
I20260809 19:27:13.389195 1758 tablet_bootstrap.cc:492] T 0592419e6e8c431ca58e30426f84c197 P 2006df36d636448ca48345eae800cdde: No bootstrap required, opened a new log
I20260809 19:27:13.389283 1758 ts_tablet_manager.cc:1397] T 0592419e6e8c431ca58e30426f84c197 P 2006df36d636448ca48345eae800cdde: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 19:27:13.392486 1758 raft_consensus.cc:348] T 0592419e6e8c431ca58e30426f84c197 P 2006df36d636448ca48345eae800cdde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c6c2a3206d4828ad23fcd650515221" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38153 } } peers { permanent_uuid: "f86102b7988a48799264c600d0e7be60" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34155 } } peers { permanent_uuid: "2006df36d636448ca48345eae800cdde" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46527 } }
I20260809 19:27:13.392668 1758 raft_consensus.cc:374] T 0592419e6e8c431ca58e30426f84c197 P 2006df36d636448ca48345eae800cdde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:27:13.392711 1758 raft_consensus.cc:729] T 0592419e6e8c431ca58e30426f84c197 P 2006df36d636448ca48345eae800cdde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2006df36d636448ca48345eae800cdde, State: Initialized, Role: FOLLOWER
I20260809 19:27:13.392863 1758 consensus_queue.cc:260] T 0592419e6e8c431ca58e30426f84c197 P 2006df36d636448ca48345eae800cdde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c6c2a3206d4828ad23fcd650515221" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38153 } } peers { permanent_uuid: "f86102b7988a48799264c600d0e7be60" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34155 } } peers { permanent_uuid: "2006df36d636448ca48345eae800cdde" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46527 } }
I20260809 19:27:13.393138 1758 ts_tablet_manager.cc:1428] T 0592419e6e8c431ca58e30426f84c197 P 2006df36d636448ca48345eae800cdde: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 19:27:13.393185 1448 heartbeater.cc:503] Master 127.21.81.254:43371 was elected leader, sending a full tablet report...
Aug 09 19:27:13 dist-test-slave-njxd krb5kdc[728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303632, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
W20260809 19:27:13.460559 1449 tablet.cc:2369] T 0592419e6e8c431ca58e30426f84c197 P 2006df36d636448ca48345eae800cdde: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:27:13.513020 1593 tablet.cc:2369] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:27:13.539721 1736 tablet.cc:2369] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:27:13.598827 1759 raft_consensus.cc:482] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:27:13.598992 1759 raft_consensus.cc:504] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c6c2a3206d4828ad23fcd650515221" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38153 } } peers { permanent_uuid: "f86102b7988a48799264c600d0e7be60" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34155 } } peers { permanent_uuid: "2006df36d636448ca48345eae800cdde" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46527 } }
I20260809 19:27:13.599408 1759 leader_election.cc:288] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f86102b7988a48799264c600d0e7be60 (127.21.81.195:34155), 2006df36d636448ca48345eae800cdde (127.21.81.193:46527)
I20260809 19:27:13.603922 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.600012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:55803 (local address 127.21.81.193:46527)
0809 19:27:13.600062 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:27:13.600064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:27:13.600075 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:27:13.600117 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:27:13.600118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:27:13.600130 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:27:13.600214 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:27:13.600530 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.601322 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.602873 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.603482 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.603535 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.603546 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 19:27:13.603796 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 19:27:13.603841 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:27:13.604161 1766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.599722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:46527 (local address 127.21.81.194:55803)
0809 19:27:13.599881 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 19:27:13.599895 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:27:13.599970 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:27:13.600243 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:27:13.600244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:27:13.600247 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:27:13.600420 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:13.600423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.601453 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.601454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:13.602760 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:13.602764 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.603622 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.603623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:13.603709 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.603713 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:27:13.604026 (+ 313us) client_negotiation.cc:240] Negotiation successful
0809 19:27:13.604076 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260809 19:27:13.604595 1400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0592419e6e8c431ca58e30426f84c197" candidate_uuid: "c1c6c2a3206d4828ad23fcd650515221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2006df36d636448ca48345eae800cdde" is_pre_election: true
I20260809 19:27:13.604782 1400 raft_consensus.cc:2454] T 0592419e6e8c431ca58e30426f84c197 P 2006df36d636448ca48345eae800cdde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1c6c2a3206d4828ad23fcd650515221 in term 0.
I20260809 19:27:13.605109 1475 leader_election.cc:302] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2006df36d636448ca48345eae800cdde, c1c6c2a3206d4828ad23fcd650515221; no voters:
I20260809 19:27:13.605266 1759 raft_consensus.cc:2781] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:27:13.605324 1759 raft_consensus.cc:482] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:27:13.605346 1759 raft_consensus.cc:3037] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:27:13.606688 1759 raft_consensus.cc:504] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c6c2a3206d4828ad23fcd650515221" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38153 } } peers { permanent_uuid: "f86102b7988a48799264c600d0e7be60" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34155 } } peers { permanent_uuid: "2006df36d636448ca48345eae800cdde" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46527 } }
I20260809 19:27:13.606846 1759 leader_election.cc:288] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [CANDIDATE]: Term 1 election: Requested vote from peers f86102b7988a48799264c600d0e7be60 (127.21.81.195:34155), 2006df36d636448ca48345eae800cdde (127.21.81.193:46527)
I20260809 19:27:13.607167 1400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0592419e6e8c431ca58e30426f84c197" candidate_uuid: "c1c6c2a3206d4828ad23fcd650515221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2006df36d636448ca48345eae800cdde"
I20260809 19:27:13.607268 1400 raft_consensus.cc:3037] T 0592419e6e8c431ca58e30426f84c197 P 2006df36d636448ca48345eae800cdde [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:27:13.607661 1751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.600592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:47117 (local address 127.21.81.195:34155)
0809 19:27:13.600632 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:27:13.600634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:27:13.604384 (+ 3750us) server_negotiation.cc:407] Connection header received
0809 19:27:13.604492 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:27:13.604493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:27:13.604505 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:27:13.604535 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:27:13.604725 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.605411 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.606751 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.607288 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.607338 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.607346 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:27:13.607538 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 19:27:13.607593 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:27:13.607844 1766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.600417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:34155 (local address 127.21.81.194:47117)
0809 19:27:13.604320 (+ 3903us) negotiation.cc:107] Waiting for socket to connect
0809 19:27:13.604332 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:27:13.604416 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:27:13.604617 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:27:13.604618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:27:13.604620 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:27:13.604650 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:13.604651 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.605560 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.605561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:13.606642 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:27:13.606645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.607404 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.607405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:27:13.607462 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.607465 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:27:13.607740 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 19:27:13.607782 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3890,"thread_start_us":71,"threads_started":1}
I20260809 19:27:13.608304 1400 raft_consensus.cc:2454] T 0592419e6e8c431ca58e30426f84c197 P 2006df36d636448ca48345eae800cdde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1c6c2a3206d4828ad23fcd650515221 in term 1.
I20260809 19:27:13.608536 1475 leader_election.cc:302] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2006df36d636448ca48345eae800cdde, c1c6c2a3206d4828ad23fcd650515221; no voters:
I20260809 19:27:13.608800 1686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0592419e6e8c431ca58e30426f84c197" candidate_uuid: "c1c6c2a3206d4828ad23fcd650515221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f86102b7988a48799264c600d0e7be60"
I20260809 19:27:13.608923 1686 raft_consensus.cc:3037] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:27:13.608812 1688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0592419e6e8c431ca58e30426f84c197" candidate_uuid: "c1c6c2a3206d4828ad23fcd650515221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f86102b7988a48799264c600d0e7be60" is_pre_election: true
I20260809 19:27:13.609598 1759 raft_consensus.cc:2781] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:27:13.609838 1759 raft_consensus.cc:686] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [term 1 LEADER]: Becoming Leader. State: Replica: c1c6c2a3206d4828ad23fcd650515221, State: Running, Role: LEADER
I20260809 19:27:13.609927 1759 consensus_queue.cc:237] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c6c2a3206d4828ad23fcd650515221" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38153 } } peers { permanent_uuid: "f86102b7988a48799264c600d0e7be60" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34155 } } peers { permanent_uuid: "2006df36d636448ca48345eae800cdde" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46527 } }
I20260809 19:27:13.609958 1686 raft_consensus.cc:2454] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1c6c2a3206d4828ad23fcd650515221 in term 1.
I20260809 19:27:13.610800 1223 catalog_manager.cc:5495] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1c6c2a3206d4828ad23fcd650515221 (127.21.81.194). New cstate: current_term: 1 leader_uuid: "c1c6c2a3206d4828ad23fcd650515221" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c6c2a3206d4828ad23fcd650515221" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f86102b7988a48799264c600d0e7be60" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2006df36d636448ca48345eae800cdde" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46527 } health_report { overall_health: UNKNOWN } } }
I20260809 19:27:13.644264 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.640575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43696 (local address 127.21.81.194:38153)
0809 19:27:13.640621 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:27:13.640623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:27:13.640651 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:27:13.640757 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:27:13.640758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:27:13.640785 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:27:13.640899 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:27:13.641102 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.641871 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.643051 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:13.643363 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:13.643432 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:13.643667 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:27:13.643815 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:27:13.643898 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 19:27:13.644150 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 19:27:13.644175 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:27:13.658890 1686 raft_consensus.cc:1261] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60 [term 1 FOLLOWER]: Refusing update from remote peer c1c6c2a3206d4828ad23fcd650515221: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:27:13.659282 1400 raft_consensus.cc:1261] T 0592419e6e8c431ca58e30426f84c197 P 2006df36d636448ca48345eae800cdde [term 1 FOLLOWER]: Refusing update from remote peer c1c6c2a3206d4828ad23fcd650515221: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:27:13.659814 1759 consensus_queue.cc:1035] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2006df36d636448ca48345eae800cdde" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:27:13.660320 1759 consensus_queue.cc:1035] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f86102b7988a48799264c600d0e7be60" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 34155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:27:13.673537 1774 mvcc.cc:205] Tried to move back new op lower bound from 7316699683463221248 to 7316699683266674688. Current Snapshot: MvccSnapshot[applied={T|T < 7316699683463221248}]
I20260809 19:27:13.675541 1772 mvcc.cc:205] Tried to move back new op lower bound from 7316699683463221248 to 7316699683266674688. Current Snapshot: MvccSnapshot[applied={T|T < 7316699683463221248}]
I20260809 19:27:13.677556 1223 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:27:13.677687 1223 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:27:13.678509 1223 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3617040d4b754e6fa86687c3f548c957: Sending DeleteTablet for 3 replicas of tablet 0592419e6e8c431ca58e30426f84c197
I20260809 19:27:13.678933 1666 tablet_service.cc:1514] Processing DeleteTablet for tablet 0592419e6e8c431ca58e30426f84c197 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:27:13 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:59736
I20260809 19:27:13.679155 1379 tablet_service.cc:1514] Processing DeleteTablet for tablet 0592419e6e8c431ca58e30426f84c197 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:27:13 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:38006
I20260809 19:27:13.679518 1525 tablet_service.cc:1514] Processing DeleteTablet for tablet 0592419e6e8c431ca58e30426f84c197 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:27:13 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:43684
I20260809 19:27:13.680158 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 1309
I20260809 19:27:13.689759 1782 tablet_replica.cc:330] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221: stopping tablet replica
I20260809 19:27:13.689975 1782 raft_consensus.cc:2229] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:27:13.690584 1780 tablet_replica.cc:330] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60: stopping tablet replica
I20260809 19:27:13.690429 1782 raft_consensus.cc:2258] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:27:13.695523 1780 raft_consensus.cc:2229] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:27:13.695789 1780 raft_consensus.cc:2258] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:27:13.696283 1782 ts_tablet_manager.cc:1905] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:27:13.699213 1780 ts_tablet_manager.cc:1905] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:27:13.701339 1780 ts_tablet_manager.cc:1918] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:27:13.702009 1782 ts_tablet_manager.cc:1918] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:27:13.702144 1782 log.cc:1196] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/0592419e6e8c431ca58e30426f84c197
I20260809 19:27:13.702529 1782 ts_tablet_manager.cc:1939] T 0592419e6e8c431ca58e30426f84c197 P c1c6c2a3206d4828ad23fcd650515221: Deleting consensus metadata
I20260809 19:27:13.702678 1780 log.cc:1196] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/0592419e6e8c431ca58e30426f84c197
I20260809 19:27:13.703231 1206 catalog_manager.cc:4841] TS c1c6c2a3206d4828ad23fcd650515221 (127.21.81.194:38153): tablet 0592419e6e8c431ca58e30426f84c197 (table test-table [id=af824e15208f4ddab0414f759525507e]) successfully deleted
I20260809 19:27:13.703730 1780 ts_tablet_manager.cc:1939] T 0592419e6e8c431ca58e30426f84c197 P f86102b7988a48799264c600d0e7be60: Deleting consensus metadata
I20260809 19:27:13.704233 1207 catalog_manager.cc:4841] TS f86102b7988a48799264c600d0e7be60 (127.21.81.195:34155): tablet 0592419e6e8c431ca58e30426f84c197 (table test-table [id=af824e15208f4ddab0414f759525507e]) successfully deleted
W20260809 19:27:13.704964 1208 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:46527 (error 108)
I20260809 19:27:13.709874 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 1456
I20260809 19:27:13.710119 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:13.709966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:46527 (local address 127.0.0.1:38010)
0809 19:27:13.710020 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 19:27:13.710054 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 19:27:13.712141 1208 catalog_manager.cc:4584] TS 2006df36d636448ca48345eae800cdde (127.21.81.193:46527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0592419e6e8c431ca58e30426f84c197: Network error: Client connection negotiation failed: client connection to 127.21.81.193:46527: connect: Connection refused (error 111)
I20260809 19:27:13.736981 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 1598
I20260809 19:27:13.744936 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 1189
I20260809 19:27:13.758823 21831 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 19:27:14.104148 21831 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 19:27:14.104257 21831 mini_ranger.cc:66] Stopping Ranger...
I20260809 19:27:14.449987 21831 mini_ranger.cc:68] Stopped Ranger
2026-08-09 19:27:14.450 UTC [759] LOG: received smart shutdown request
2026-08-09 19:27:14.453 UTC [759] LOG: background worker "logical replication launcher" (PID 768) exited with exit code 1
2026-08-09 19:27:14.453 UTC [763] LOG: shutting down
2026-08-09 19:27:14.603 UTC [759] LOG: database system is shut down
2026-08-09T19:27:14Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49691 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:27:14 dist-test-slave-njxd krb5kdc[1794](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:27:14 dist-test-slave-njxd krb5kdc[1794](info): set up 2 sockets
Aug 09 19:27:14 dist-test-slave-njxd krb5kdc[1794](info): commencing operation
krb5kdc: starting...
W20260809 19:27:15.796059 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:27:15 dist-test-slave-njxd krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:27:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:27:15Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.81.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.81.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/rangeradmin_127.21.81.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/rangeradmin_127.21.81.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.81.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.81.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/rangerlookup_127.21.81.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/rangerlookup_127.21.81.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.81.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.81.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/HTTP_127.21.81.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/HTTP_127.21.81.212@KRBTEST.COM.keytab.
I20260809 19:27:15.956005 21831 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskkvvQ95/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/postgres -l logfile start
2026-08-09 19:27:18.276 UTC [1825] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 19:27:18.276 UTC [1825] LOG: listening on IPv4 address "127.21.81.212", port 47895
2026-08-09 19:27:18.279 UTC [1825] LOG: listening on Unix socket "/tmp/.s.PGSQL.47895"
2026-08-09 19:27:18.314 UTC [1828] LOG: database system was shut down at 2026-08-09 19:27:17 UTC
2026-08-09 19:27:18.318 UTC [1825] LOG: database system is ready to accept connections
I20260809 19:27:19.288545 21831 mini_postgres.cc:96] Postgres bound to 47895
2026-08-09 19:27:19.295 UTC [1836] FATAL: database "slave" does not exist
127.21.81.212:47895 - accepting connections
I20260809 19:27:19.296278 21831 mini_ranger.cc:161] Starting Ranger...
I20260809 19:27:19.311606 21831 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 19:27:19.746394 21831 mini_ranger.cc:87] Created ranger Postgres database
I20260809 19:27:19.746485 21831 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-admin
2026-08-09 19:27:20,026 [I] DB FLAVOR :POSTGRES
2026-08-09 19:27:20,026 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 19:27:20,026 [I] Checking connection..
2026-08-09 19:27:20,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 19:27:20,290 [I] Checking connection passed.
2026-08-09 19:27:20,290 [I] --------- Verifying version history table ---------
2026-08-09 19:27:20,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:27:20,556 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 19:27:20,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:27:20,797 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 19:27:20,797 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 19:27:20,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 19:27:21.056 UTC [1902] WARNING: there is no transaction in progress
2026-08-09 19:27:21,064 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 19:27:21,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:27:21,330 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 19:27:21,330 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 19:27:21,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 19:27:21,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 19:27:21,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-njxd.c.gcp-upstream.internal','N') ;"
2026-08-09 19:27:22,100 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 19:27:22,349 [I] Table x_portal_user does not exist in database ranger
2026-08-09 19:27:22,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 19:27:22,616 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 19:27:22,616 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 19:27:22,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 19:27:23,137 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 19:27:23,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 19:27:24.067 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24.082 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24.098 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24.111 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24.126 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24.141 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24.192 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24.198 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24.211 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24.221 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24.230 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24.244 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24.253 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24.259 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24.266 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24.272 UTC [2018] WARNING: there is no transaction in progress
2026-08-09 19:27:24,874 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 19:27:24,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 19:27:25,100 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 19:27:25,100 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 19:27:25,374 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 19:27:25,375 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 19:27:25,642 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 19:27:25,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 19:27:25,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 19:27:26,155 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-njxd.c.gcp-upstream.internal';"
2026-08-09 19:27:26,405 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 19:27:26,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 19:27:26,665 [I] DB_PATCHES have already been applied
I20260809 19:27:26.671468 21831 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-admin:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskkvvQ95/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 19:27:26.671540 21831 mini_ranger.cc:193] Using host: 127.21.81.212
I20260809 19:27:26.675315 21831 mini_ranger.cc:227] Ranger admin URL: http://127.21.81.212:57325
Aug 09, 2026 7:27:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:27:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:27:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 7:27:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 7:27:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 7:27:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 7:27:27 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 19:27:27.815217 21831 mini_ranger.cc:160] Time spent starting Ranger: real 8.519s user 0.000s sys 0.011s
Aug 09, 2026 7:27:48 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 7:27:48 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 7:27:48 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 7:27:48 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 7:27:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 7:27:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 7:27:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 7:27:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 7:27:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 7:27:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 7:27:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 7:27:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 7:27:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 7:27:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 7:27:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 7:27:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 7:27:48 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 7:27:49 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 19:27:50.393788 21831 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.81.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.81.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/rangerkms_127.21.81.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.81.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/rangerkms_127.21.81.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.81.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.81.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.81.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/HTTP_127.21.81.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.81.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/HTTP_127.21.81.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 19:27:50 dist-test-slave-njxd krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303670, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 19:27:50.482056 21831 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 19:27:50.492129 21831 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 19:27:51.251471 21831 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 19:27:51.251564 21831 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-kms
I20260809 19:27:51.251585 21831 mini_ranger_kms.cc:227] Using postgres at 127.21.81.212:47895
2026-08-09 19:27:51,426 [I] DB FLAVOR :POSTGRES
2026-08-09 19:27:51,427 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 19:27:51,427 [I] Checking connection
2026-08-09 19:27:51,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 19:27:51,666 [I] connection success
2026-08-09 19:27:51,666 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 19:27:51,667 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 19:27:51,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 19:27:51,930 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 19:27:51,930 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 19:27:51,930 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 19:27:51,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.81.212:47895/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 19:27:52,190 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 19:27:52.678447 21831 mini_ranger_kms.cc:310] Created kms service
I20260809 19:27:52.899222 21831 mini_ranger_kms.cc:326] Created kudu user
I20260809 19:27:52.949784 21831 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 19:27:52 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303670, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
I20260809 19:27:53.356081 21831 mini_ranger_kms.cc:384] Added ranger policy
I20260809 19:27:53.356328 21831 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-kms:/tmp/dist-test-taskkvvQ95/build/release/bin/postgresql.jar:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskkvvQ95/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 19:27:53.356351 21831 mini_ranger_kms.cc:240] Using host: 127.21.81.212
I20260809 19:27:53.360543 21831 mini_ranger_kms.cc:276] Ranger KMS PID: 2208
I20260809 19:27:53.360634 21831 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.81.212:36843
Aug 09, 2026 7:27:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:27:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:27:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 7:27:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 7:27:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 7:27:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 19:27:55.756970 21831 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.275s user 0.000s sys 0.159s
I20260809 19:27:55.757063 21831 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 19:27:55.757120 21831 mini_ranger_kms.cc:401] 127.21.81.212:36843/kms/v1/keys
Aug 09 19:27:56 dist-test-slave-njxd krb5kdc[1794](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786303676, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.81.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 19:27:56 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303676, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.81.212@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
Aug 09 19:27:56 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303676, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.81.212@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
Aug 09 19:27:59 dist-test-slave-njxd krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:27:59.098438 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:42829
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:42829
--ranger_config_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.81.212:36843/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:27:59.181865 2260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:27:59.182060 2260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:27:59.182080 2260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:27:59.183319 2260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:27:59.183369 2260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:27:59.183383 2260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:27:59.183396 2260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:27:59.183408 2260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:27:59.184939 2260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.81.212:36843/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:42829
--ranger_config_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:42829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:27:59.185178 2260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:27:59.185425 2260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:27:59.191642 2266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:27:59.191684 2260 server_base.cc:1034] running on GCE node
W20260809 19:27:59.191653 2268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:27:59.191890 2265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:27:59.192181 2260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:27:59.192534 2260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:27:59.193713 2260 hybrid_clock.cc:648] HybridClock initialized: now 1786303679193677 us; error 43 us; skew 500 ppm
Aug 09 19:27:59 dist-test-slave-njxd krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303679, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:27:59.196851 2260 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:27:59.197384 2260 webserver.cc:460] Webserver started at http://127.21.81.254:34767/ using document root <none> and password file <none>
I20260809 19:27:59.197708 2260 fs_manager.cc:359] Metadata directory not provided
I20260809 19:27:59.197772 2260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:27:59.197921 2260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:27:59 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303679, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
I20260809 19:27:59.255069 2260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "fd8c3573298444099d882f998e785f8e"
format_stamp: "Formatted at 2026-08-09 19:27:59 on dist-test-slave-njxd"
server_key: "046bf449b3afcba6f25a97c78ec5fb0a"
server_key_iv: "ad2fd93a0dc0f44128e8a38d0accc1a3"
server_key_version: "kuduclusterkey@0"
I20260809 19:27:59.255530 2260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "fd8c3573298444099d882f998e785f8e"
format_stamp: "Formatted at 2026-08-09 19:27:59 on dist-test-slave-njxd"
server_key: "046bf449b3afcba6f25a97c78ec5fb0a"
server_key_iv: "ad2fd93a0dc0f44128e8a38d0accc1a3"
server_key_version: "kuduclusterkey@0"
I20260809 19:27:59.257019 2260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:27:59.270026 2275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:27:59.270260 2260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:27:59.270368 2260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "fd8c3573298444099d882f998e785f8e"
format_stamp: "Formatted at 2026-08-09 19:27:59 on dist-test-slave-njxd"
server_key: "046bf449b3afcba6f25a97c78ec5fb0a"
server_key_iv: "ad2fd93a0dc0f44128e8a38d0accc1a3"
server_key_version: "kuduclusterkey@0"
I20260809 19:27:59.270495 2260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:27:59.299834 2260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:27:59.302081 2260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:27:59.302268 2260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:27:59.307391 2260 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:42829
I20260809 19:27:59.307425 2332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:42829 every 8 connection(s)
I20260809 19:27:59.307788 2260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:27:59.308513 2333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:27:59.310877 2333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e: Bootstrap starting.
I20260809 19:27:59.311496 2333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e: Neither blocks nor log segments found. Creating new log.
I20260809 19:27:59.311805 2333 log.cc:824] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e: Log is configured to *not* fsync() on all Append() calls
I20260809 19:27:59.312515 2333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e: No bootstrap required, opened a new log
I20260809 19:27:59.314659 2333 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8c3573298444099d882f998e785f8e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42829 } }
I20260809 19:27:59.314811 2333 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:27:59.314836 2333 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8c3573298444099d882f998e785f8e, State: Initialized, Role: FOLLOWER
I20260809 19:27:59.314989 2333 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8c3573298444099d882f998e785f8e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42829 } }
I20260809 19:27:59.315065 2333 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:27:59.315088 2333 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:27:59.315155 2333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:27:59.315407 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 2260
I20260809 19:27:59.315523 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:27:59.315708 21831 external_mini_cluster.cc:1428] Setting key 2e41de639985e18cd870bdeda4efd120
I20260809 19:27:59.315866 2333 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8c3573298444099d882f998e785f8e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42829 } }
I20260809 19:27:59.316000 2333 leader_election.cc:302] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd8c3573298444099d882f998e785f8e; no voters:
I20260809 19:27:59.316190 2333 leader_election.cc:288] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:27:59.316485 2333 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:27:59.316524 2336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:27:59.316720 2336 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 1 LEADER]: Becoming Leader. State: Replica: fd8c3573298444099d882f998e785f8e, State: Running, Role: LEADER
I20260809 19:27:59.316823 2336 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8c3573298444099d882f998e785f8e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42829 } }
I20260809 19:27:59.317112 2336 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd8c3573298444099d882f998e785f8e. Latest consensus state: current_term: 1 leader_uuid: "fd8c3573298444099d882f998e785f8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8c3573298444099d882f998e785f8e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42829 } } }
I20260809 19:27:59.317188 2336 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [sys.catalog]: This master's current role is: LEADER
I20260809 19:27:59.317473 2339 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd8c3573298444099d882f998e785f8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8c3573298444099d882f998e785f8e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42829 } } }
I20260809 19:27:59.317610 2339 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [sys.catalog]: This master's current role is: LEADER
I20260809 19:27:59.317833 2346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:27:59.318137 2333 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 19:27:59.318395 2346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:27:59.324913 2356 mvcc.cc:205] Tried to move back new op lower bound from 7316699870511644672 to 7316699870481924096. Current Snapshot: MvccSnapshot[applied={T|T < 7316699870511644672}]
I20260809 19:27:59.325690 2346 catalog_manager.cc:1347] Generated new cluster ID: 0fbd2b3353434332b0757e5baac9c6f5
I20260809 19:27:59.325759 2346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:27:59 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:27:59.331902 2338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:27:59.316225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60058 (local address 127.21.81.254:42829)
0809 19:27:59.316491 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 19:27:59.316501 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:27:59.316535 (+ 34us) server_negotiation.cc:407] Connection header received
0809 19:27:59.316656 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:27:59.316663 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:27:59.316749 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:27:59.317761 (+ 1012us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:27:59.317876 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:59.321480 (+ 3604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:59.322764 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:27:59.323749 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:27:59.326851 (+ 3102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:27:59.327021 (+ 170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:27:59.327024 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:27:59.327024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:27:59.327034 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:27:59.329620 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:27:59.330048 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:27:59.330051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:27:59.330052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:27:59.330187 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:27:59.330393 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:27:59.330394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:27:59.330394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:27:59.330745 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:27:59.331352 (+ 607us) server_negotiation.cc:1035] Waiting for connection context
0809 19:27:59.331428 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 19:27:59.331605 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":86,"threads_started":1}
I20260809 19:27:59.341786 2346 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:27:59.342607 2346 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:27:59.349512 2346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e: Generated new TSK 0
I20260809 19:27:59.349858 2346 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:28:00.017103 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.81.212:36843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42829
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:00.135185 2380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:00.135412 2380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:00.135452 2380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:00.136626 2380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:00.136693 2380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:00.136736 2380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:28:00.138257 2380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.81.212:36843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:00.138530 2380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:00.138758 2380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:00.143854 2386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:00.144204 2385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:00.144362 2380 server_base.cc:1034] running on GCE node
W20260809 19:28:00.144616 2388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:00.144899 2380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:00.145161 2380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:00.146322 2380 hybrid_clock.cc:648] HybridClock initialized: now 1786303680146301 us; error 42 us; skew 500 ppm
Aug 09 19:28:00 dist-test-slave-njxd krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303680, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:00.149133 2380 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:28:00.149705 2380 webserver.cc:460] Webserver started at http://127.21.81.193:36339/ using document root <none> and password file <none>
I20260809 19:28:00.149973 2380 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:00.150034 2380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:00.150166 2380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:28:00 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303680, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
I20260809 19:28:00.163777 2380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "a4d663b5cba841089b8caa1d9799db89"
format_stamp: "Formatted at 2026-08-09 19:28:00 on dist-test-slave-njxd"
server_key: "dacab32a3e80d32fd0a1f8047a3136be"
server_key_iv: "9c8f3a2785fd13d90c31757d9dcad62a"
server_key_version: "kuduclusterkey@0"
I20260809 19:28:00.164314 2380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "a4d663b5cba841089b8caa1d9799db89"
format_stamp: "Formatted at 2026-08-09 19:28:00 on dist-test-slave-njxd"
server_key: "dacab32a3e80d32fd0a1f8047a3136be"
server_key_iv: "9c8f3a2785fd13d90c31757d9dcad62a"
server_key_version: "kuduclusterkey@0"
I20260809 19:28:00.169989 2380 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.001s
I20260809 19:28:00.183622 2396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:00.183871 2380 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:28:00.183964 2380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "a4d663b5cba841089b8caa1d9799db89"
format_stamp: "Formatted at 2026-08-09 19:28:00 on dist-test-slave-njxd"
server_key: "dacab32a3e80d32fd0a1f8047a3136be"
server_key_iv: "9c8f3a2785fd13d90c31757d9dcad62a"
server_key_version: "kuduclusterkey@0"
I20260809 19:28:00.184057 2380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:00.197873 2380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:00.200506 2380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:00.200736 2380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:00.201059 2380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:00.201501 2380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:00.201627 2380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:00.201741 2380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:00.201825 2380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:00.210367 2380 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:41845
I20260809 19:28:00.210477 2517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:41845 every 8 connection(s)
I20260809 19:28:00.210934 2380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:28:00.213760 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 2380
I20260809 19:28:00.213871 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:28:00.214075 21831 external_mini_cluster.cc:1428] Setting key f0e0990014aaf905fa8bd22e501b1c94
Aug 09 19:28:00 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303680, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:00.222254 2521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:00.211847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:49629 (local address 127.21.81.254:42829)
0809 19:28:00.212437 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:00.212439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:00.212451 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:28:00.212804 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:00.212805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:00.212820 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:00.212900 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:00.213470 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:00.214077 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:00.215482 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:00.215705 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:00.215791 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:00.218480 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:00.218481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:00.218482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:00.218490 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:00.220514 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:00.221027 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:00.221029 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:00.221029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:00.221111 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:00.221453 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:00.221454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:00.221455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:00.221621 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:00.221674 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:00.222063 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 19:28:00.222152 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":80,"threads_started":1}
I20260809 19:28:00.222821 2520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:00.211799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42829 (local address 127.21.81.193:49629)
0809 19:28:00.211999 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:00.212051 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:00.212686 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:00.212976 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:00.212979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:00.213175 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:00.213364 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:00.213368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:00.214172 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:00.214173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:00.215373 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:00.215376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:00.215817 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:00.215819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:00.215998 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:00.216543 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:00.216551 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:00.218382 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:00.220613 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:00.220620 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:00.220622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:00.220941 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:00.221225 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:00.221226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:00.221226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:00.221362 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:00.221728 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:00.221734 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:00.221895 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 19:28:00.222400 (+ 505us) client_negotiation.cc:240] Negotiation successful
0809 19:28:00.222580 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
I20260809 19:28:00.225040 2518 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42829
I20260809 19:28:00.225140 2518 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:00.225337 2518 heartbeater.cc:511] Master 127.21.81.254:42829 requested a full tablet report, sending...
I20260809 19:28:00.225903 2294 ts_manager.cc:194] Registered new tserver with Master: a4d663b5cba841089b8caa1d9799db89 (127.21.81.193:41845)
I20260809 19:28:00.226864 2294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:49629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:28:00.244688 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.81.212:36843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42829
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:00.361222 2524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:00.361518 2524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:00.361559 2524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:00.363291 2524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:00.363354 2524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:00.363394 2524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:28:00.365494 2524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.81.212:36843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:00.365756 2524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:00.366079 2524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:00.372510 2531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:00.372710 2532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:00.372961 2534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:00.373513 2524 server_base.cc:1034] running on GCE node
I20260809 19:28:00.373879 2524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:00.374259 2524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:00.375452 2524 hybrid_clock.cc:648] HybridClock initialized: now 1786303680375414 us; error 51 us; skew 500 ppm
Aug 09 19:28:00 dist-test-slave-njxd krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303680, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:00.378785 2524 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:28:00.379406 2524 webserver.cc:460] Webserver started at http://127.21.81.194:39755/ using document root <none> and password file <none>
I20260809 19:28:00.379961 2524 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:00.380100 2524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:00.380292 2524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:28:00 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303680, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
Aug 09 19:28:00 dist-test-slave-njxd krb5kdc[1794](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786303680, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:00.395370 2524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "422a92aefca74494815c5a64243df097"
format_stamp: "Formatted at 2026-08-09 19:28:00 on dist-test-slave-njxd"
server_key: "94ba45a7a410529f53145da9bf5dea82"
server_key_iv: "d04a7e82ccd2c74dcc659e06c693965a"
server_key_version: "kuduclusterkey@0"
I20260809 19:28:00.398092 2524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "422a92aefca74494815c5a64243df097"
format_stamp: "Formatted at 2026-08-09 19:28:00 on dist-test-slave-njxd"
server_key: "94ba45a7a410529f53145da9bf5dea82"
server_key_iv: "d04a7e82ccd2c74dcc659e06c693965a"
server_key_version: "kuduclusterkey@0"
I20260809 19:28:00.406076 2524 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.000s
I20260809 19:28:00.429924 2542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:00.430212 2524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 19:28:00.430323 2524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "422a92aefca74494815c5a64243df097"
format_stamp: "Formatted at 2026-08-09 19:28:00 on dist-test-slave-njxd"
server_key: "94ba45a7a410529f53145da9bf5dea82"
server_key_iv: "d04a7e82ccd2c74dcc659e06c693965a"
server_key_version: "kuduclusterkey@0"
I20260809 19:28:00.430445 2524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:00.445015 2524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:00.447125 2524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:00.447268 2524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:00.447520 2524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:00.447863 2524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:00.447907 2524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:00.447938 2524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:00.447965 2524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:00.454898 2524 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:33913
I20260809 19:28:00.454983 2662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:33913 every 8 connection(s)
I20260809 19:28:00.455309 2524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:28:00 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303680, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:00.464653 2521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:00.456076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:36645 (local address 127.21.81.254:42829)
0809 19:28:00.456151 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:00.456155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:00.456369 (+ 214us) server_negotiation.cc:407] Connection header received
0809 19:28:00.456854 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:00.456855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:00.456866 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:00.456946 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:00.457448 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:00.458274 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:00.459243 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:00.459562 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:00.459629 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:00.461890 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:00.461892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:00.461892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:00.461898 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:00.463225 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:00.463708 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:00.463709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:00.463709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:00.463761 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:00.464035 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:00.464036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:00.464036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:00.464134 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:00.464183 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:00.464516 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 19:28:00.464587 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 19:28:00.464941 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:00.456056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42829 (local address 127.21.81.194:36645)
0809 19:28:00.456237 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:00.456279 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:00.456766 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:00.456998 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:00.457001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:00.457179 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:00.457373 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:00.457377 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:00.458369 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:00.458370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:00.459164 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:00.459166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:00.459652 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:00.459653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:00.459806 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:00.460305 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:00.460313 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:00.461796 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:00.463313 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:00.463320 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:00.463322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:00.463623 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:00.463823 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:00.463824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:00.463825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:00.463965 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:00.464205 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:00.464212 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:00.464370 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 19:28:00.464489 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 19:28:00.464644 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":110,"threads_started":1}
I20260809 19:28:00.465404 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 2524
I20260809 19:28:00.465456 2663 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42829
I20260809 19:28:00.465512 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:28:00.465555 2663 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:00.465759 2663 heartbeater.cc:511] Master 127.21.81.254:42829 requested a full tablet report, sending...
I20260809 19:28:00.465761 21831 external_mini_cluster.cc:1428] Setting key be906f8d8e3a78b5793e77839577c0a8
I20260809 19:28:00.466282 2293 ts_manager.cc:194] Registered new tserver with Master: 422a92aefca74494815c5a64243df097 (127.21.81.194:33913)
I20260809 19:28:00.466704 2293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:36645
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:28:00.496578 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.81.212:36843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42829
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:00.611912 2669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:00.612216 2669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:00.612262 2669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:00.614053 2669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:00.614118 2669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:00.614159 2669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:28:00.616413 2669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.81.212:36843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:42829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:00.616677 2669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:00.616968 2669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:00.623505 2675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:00.623883 2677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:00.624059 2674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:00.625203 2669 server_base.cc:1034] running on GCE node
I20260809 19:28:00.625423 2669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:00.625735 2669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:00.626925 2669 hybrid_clock.cc:648] HybridClock initialized: now 1786303680626903 us; error 32 us; skew 500 ppm
Aug 09 19:28:00 dist-test-slave-njxd krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303680, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:00.630342 2669 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:28:00.630946 2669 webserver.cc:460] Webserver started at http://127.21.81.195:46421/ using document root <none> and password file <none>
I20260809 19:28:00.631242 2669 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:00.631313 2669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:00.631453 2669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:28:00 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303680, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
I20260809 19:28:00.651283 2669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd"
format_stamp: "Formatted at 2026-08-09 19:28:00 on dist-test-slave-njxd"
server_key: "2ccb49a2415c0bed44622d4f6478cc79"
server_key_iv: "7d7d0c9886e1b83a74dd6d46b7c9f141"
server_key_version: "kuduclusterkey@0"
I20260809 19:28:00.651764 2669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd"
format_stamp: "Formatted at 2026-08-09 19:28:00 on dist-test-slave-njxd"
server_key: "2ccb49a2415c0bed44622d4f6478cc79"
server_key_iv: "7d7d0c9886e1b83a74dd6d46b7c9f141"
server_key_version: "kuduclusterkey@0"
I20260809 19:28:00.653846 2669 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260809 19:28:00.668780 2684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:00.669056 2669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:00.669162 2669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd"
format_stamp: "Formatted at 2026-08-09 19:28:00 on dist-test-slave-njxd"
server_key: "2ccb49a2415c0bed44622d4f6478cc79"
server_key_iv: "7d7d0c9886e1b83a74dd6d46b7c9f141"
server_key_version: "kuduclusterkey@0"
I20260809 19:28:00.669262 2669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:00.689061 2669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:00.702311 2669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:00.702564 2669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:00.703055 2669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:00.703729 2669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:00.703809 2669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:00.703876 2669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:00.703903 2669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:00.717767 2669 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:44147
I20260809 19:28:00.718154 2669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:28:00.719084 2811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:44147 every 8 connection(s)
I20260809 19:28:00.724483 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 2669
I20260809 19:28:00.724606 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:28:00.724810 21831 external_mini_cluster.cc:1428] Setting key 06e163886b7621c76e4807654e52e653
Aug 09 19:28:00 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303680, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:00.731953 2521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:00.719827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:48713 (local address 127.21.81.254:42829)
0809 19:28:00.719881 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:00.719883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:00.721304 (+ 1421us) server_negotiation.cc:407] Connection header received
0809 19:28:00.721973 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:00.721975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:00.721988 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:00.722042 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:00.722694 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:00.723252 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:00.724413 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:00.724740 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:00.724791 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:00.727611 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:00.727613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:00.727613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:00.727621 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:00.729399 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:00.729996 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:00.729997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:00.729998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:00.730064 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:00.730381 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:00.730383 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:00.730383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:00.730919 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:00.730988 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:00.731751 (+ 763us) server_negotiation.cc:299] Negotiation successful
0809 19:28:00.731850 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:28:00.734444 2814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:00.719832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42829 (local address 127.21.81.195:48713)
0809 19:28:00.721139 (+ 1307us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:00.721181 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:00.721863 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:00.722126 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:00.722129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:00.722385 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:00.722589 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:00.722594 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:00.723365 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:00.723367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:00.724293 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:00.724297 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:00.724976 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:00.724977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:00.725131 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:00.725874 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:00.725885 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:00.727473 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:00.729502 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:00.729509 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:00.729512 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:00.729885 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:00.730146 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:00.730146 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:00.730147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:00.730291 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:00.731323 (+ 1032us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:00.731349 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:00.731536 (+ 187us) client_negotiation.cc:769] Sending connection context
0809 19:28:00.733807 (+ 2271us) client_negotiation.cc:240] Negotiation successful
0809 19:28:00.734018 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1262,"thread_start_us":166,"threads_started":1}
I20260809 19:28:00.739169 2812 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42829
I20260809 19:28:00.739302 2812 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:00.739543 2812 heartbeater.cc:511] Master 127.21.81.254:42829 requested a full tablet report, sending...
I20260809 19:28:00.739974 2293 ts_manager.cc:194] Registered new tserver with Master: af1cf1ead4de4bd2abcd6c8a9838a2fd (127.21.81.195:44147)
I20260809 19:28:00.740489 2293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:48713
I20260809 19:28:00.748513 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:28:00.749228 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 2380
I20260809 19:28:00.757627 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 2524
I20260809 19:28:00.765265 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 2669
I20260809 19:28:00.773389 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 2260
I20260809 19:28:00.780854 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:42829
--webserver_interface=127.21.81.254
--webserver_port=34767
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:42829
--ranger_config_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.81.212:36843/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.81.212:36843/kms with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:00.862689 2815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:00.862927 2815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:00.862967 2815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:00.864120 2815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:28:00.864188 2815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:00.864223 2815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:00.864255 2815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:28:00.864279 2815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:28:00.865993 2815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.81.212:36843/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:42829
--ranger_config_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:42829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=34767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:00.866281 2815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:00.866501 2815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:00.871254 2823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:00.871209 2821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:00.871209 2820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:00.871620 2815 server_base.cc:1034] running on GCE node
I20260809 19:28:00.871850 2815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:00.872129 2815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:00.873297 2815 hybrid_clock.cc:648] HybridClock initialized: now 1786303680873283 us; error 45 us; skew 500 ppm
Aug 09 19:28:00 dist-test-slave-njxd krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303680, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:00.875968 2815 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:28:00.876434 2815 webserver.cc:460] Webserver started at http://127.21.81.254:34767/ using document root <none> and password file <none>
I20260809 19:28:00.876642 2815 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:00.876693 2815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 19:28:00 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303680, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
I20260809 19:28:00.916901 2815 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:00.917683 2831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:00.917891 2815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:00.918002 2815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "fd8c3573298444099d882f998e785f8e"
format_stamp: "Formatted at 2026-08-09 19:27:59 on dist-test-slave-njxd"
server_key: "046bf449b3afcba6f25a97c78ec5fb0a"
server_key_iv: "ad2fd93a0dc0f44128e8a38d0accc1a3"
server_key_version: "kuduclusterkey@0"
I20260809 19:28:00.918381 2815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:00.942720 2815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:00.944758 2815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:00.944941 2815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:00.949498 2815 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:42829
I20260809 19:28:00.949537 2888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:42829 every 8 connection(s)
I20260809 19:28:00.949945 2815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:28:00.951174 2890 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:28:00.951865 2890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e: Bootstrap starting.
I20260809 19:28:00.956830 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 2815
I20260809 19:28:00.957137 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.81.212:36843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:41845
--local_ip_for_outbound_sockets=127.21.81.193
--tserver_master_addrs=127.21.81.254:42829
--webserver_port=36339
--webserver_interface=127.21.81.193
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.81.212:36843/kms with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:28:00.960371 2890 log.cc:824] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:00.962337 2890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:28:00.962641 2890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e: Bootstrap complete.
I20260809 19:28:00.965122 2890 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8c3573298444099d882f998e785f8e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42829 } }
I20260809 19:28:00.965374 2890 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8c3573298444099d882f998e785f8e, State: Initialized, Role: FOLLOWER
I20260809 19:28:00.965566 2890 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8c3573298444099d882f998e785f8e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42829 } }
I20260809 19:28:00.965711 2890 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:28:00.965766 2890 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:28:00.965814 2890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:28:00.966801 2890 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8c3573298444099d882f998e785f8e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42829 } }
I20260809 19:28:00.966921 2890 leader_election.cc:302] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd8c3573298444099d882f998e785f8e; no voters:
I20260809 19:28:00.967132 2890 leader_election.cc:288] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 19:28:00.967170 2894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:28:00.967298 2894 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [term 2 LEADER]: Becoming Leader. State: Replica: fd8c3573298444099d882f998e785f8e, State: Running, Role: LEADER
I20260809 19:28:00.967386 2894 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8c3573298444099d882f998e785f8e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42829 } }
I20260809 19:28:00.967424 2890 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:28:00.967705 2894 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd8c3573298444099d882f998e785f8e. Latest consensus state: current_term: 2 leader_uuid: "fd8c3573298444099d882f998e785f8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8c3573298444099d882f998e785f8e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42829 } } }
I20260809 19:28:00.967774 2894 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:00.967713 2895 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fd8c3573298444099d882f998e785f8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8c3573298444099d882f998e785f8e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 42829 } } }
I20260809 19:28:00.967823 2895 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:00.968101 2898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:28:00.968622 2898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:28:00.968761 2898 catalog_manager.cc:1259] Loaded cluster ID: 0fbd2b3353434332b0757e5baac9c6f5
I20260809 19:28:00.968811 2898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:28:00.969317 2890 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 19:28:00.972988 2898 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:28:00.973343 2898 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e: Loaded TSK: 0
I20260809 19:28:00.973667 2898 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 19:28:01.074251 2892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:01.074525 2892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:01.074599 2892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:01.076299 2892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:01.076362 2892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:01.076411 2892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:28:01.078595 2892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.81.212:36843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:41845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=36339
--tserver_master_addrs=127.21.81.254:42829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:01.078876 2892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:01.079133 2892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:01.085824 2927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:01.085901 2928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:01.086079 2892 server_base.cc:1034] running on GCE node
W20260809 19:28:01.086289 2930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:01.086586 2892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:01.086855 2892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:01.088042 2892 hybrid_clock.cc:648] HybridClock initialized: now 1786303681088008 us; error 45 us; skew 500 ppm
Aug 09 19:28:01 dist-test-slave-njxd krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303681, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:01.091384 2892 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:28:01.091924 2892 webserver.cc:460] Webserver started at http://127.21.81.193:36339/ using document root <none> and password file <none>
I20260809 19:28:01.092187 2892 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:01.092279 2892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 19:28:01 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303681, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
I20260809 19:28:01.113991 2892 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260809 19:28:01.115787 2939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:01.115990 2892 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 19:28:01.116106 2892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "a4d663b5cba841089b8caa1d9799db89"
format_stamp: "Formatted at 2026-08-09 19:28:00 on dist-test-slave-njxd"
server_key: "dacab32a3e80d32fd0a1f8047a3136be"
server_key_iv: "9c8f3a2785fd13d90c31757d9dcad62a"
server_key_version: "kuduclusterkey@0"
I20260809 19:28:01.116595 2892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:01.134016 2892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:01.136173 2892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:01.136291 2892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:01.136524 2892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:01.136893 2892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:01.136930 2892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:01.136952 2892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:01.136968 2892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:01.144886 2892 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:41845
I20260809 19:28:01.145355 2892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:28:01.146104 3060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:41845 every 8 connection(s)
I20260809 19:28:01.153908 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 2892
I20260809 19:28:01.154251 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.81.212:36843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:33913
--local_ip_for_outbound_sockets=127.21.81.194
--tserver_master_addrs=127.21.81.254:42829
--webserver_port=39755
--webserver_interface=127.21.81.194
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.81.212:36843/kms with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:28:01 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303681, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:01.170926 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:01.146771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42829 (local address 127.21.81.193:39951)
0809 19:28:01.147208 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:01.147259 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:01.147858 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:01.158353 (+ 10495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:01.158359 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:01.158648 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:01.158966 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:01.158974 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:01.160761 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:01.160762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:01.162279 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:01.162284 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:01.163409 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:01.163410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:01.163562 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:01.164264 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:01.164275 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:01.165939 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:01.168666 (+ 2727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:01.168674 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:01.168677 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:01.169025 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:01.169417 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:01.169418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:01.169419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:01.169592 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:01.170189 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:01.170197 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:01.170379 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 19:28:01.170494 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 19:28:01.170678 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":91,"threads_started":1}
I20260809 19:28:01.171736 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:01.146869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:39951 (local address 127.21.81.254:42829)
0809 19:28:01.157713 (+ 10844us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:01.157726 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:01.157785 (+ 59us) server_negotiation.cc:407] Connection header received
0809 19:28:01.157967 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:01.157976 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:01.158050 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:01.158260 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:01.159149 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:01.160594 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:01.162408 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:01.163134 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:01.163239 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:01.166106 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:01.166111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:01.166111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:01.166126 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:01.168549 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:01.169154 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:01.169158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:01.169159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:01.169322 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:01.169687 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:01.169689 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:01.169689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:01.170041 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:01.170127 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:01.171398 (+ 1271us) server_negotiation.cc:299] Negotiation successful
0809 19:28:01.171532 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10759,"thread_start_us":118,"threads_started":1}
I20260809 19:28:01.173077 3061 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42829
I20260809 19:28:01.173174 3061 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:01.173367 3061 heartbeater.cc:511] Master 127.21.81.254:42829 requested a full tablet report, sending...
W20260809 19:28:01.174274 3061 heartbeater.cc:644] Failed to heartbeat to 127.21.81.254:42829 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 19:28:01.174345 3061 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:01.174405 3061 heartbeater.cc:511] Master 127.21.81.254:42829 requested a full tablet report, sending...
I20260809 19:28:01.174816 3061 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:01.174892 3061 heartbeater.cc:511] Master 127.21.81.254:42829 requested a full tablet report, sending...
W20260809 19:28:01.175215 3061 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:28:01.266680 3065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:01.266938 3065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:01.266978 3065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:01.268368 3065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:01.268477 3065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:01.268525 3065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:28:01.270284 3065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.81.212:36843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:33913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=39755
--tserver_master_addrs=127.21.81.254:42829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:01.270570 3065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:01.270794 3065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:01.275897 3070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:01.275887 3073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:01.276805 3065 server_base.cc:1034] running on GCE node
W20260809 19:28:01.277051 3071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:01.277309 3065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:01.277611 3065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:01.278802 3065 hybrid_clock.cc:648] HybridClock initialized: now 1786303681278785 us; error 26 us; skew 500 ppm
Aug 09 19:28:01 dist-test-slave-njxd krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303681, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:01.281514 3065 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:28:01.282095 3065 webserver.cc:460] Webserver started at http://127.21.81.194:39755/ using document root <none> and password file <none>
I20260809 19:28:01.282387 3065 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:01.282475 3065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 19:28:01 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303681, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
I20260809 19:28:01.300164 3065 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:01.301339 3081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:01.301654 3065 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:01.301779 3065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "422a92aefca74494815c5a64243df097"
format_stamp: "Formatted at 2026-08-09 19:28:00 on dist-test-slave-njxd"
server_key: "94ba45a7a410529f53145da9bf5dea82"
server_key_iv: "d04a7e82ccd2c74dcc659e06c693965a"
server_key_version: "kuduclusterkey@0"
I20260809 19:28:01.302273 3065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:01.324172 3065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:01.326658 3065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:01.326848 3065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:01.327245 3065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:01.327672 3065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:01.327731 3065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:01.327765 3065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:01.327791 3065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:01.338924 3065 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:33913
I20260809 19:28:01.338958 3202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:33913 every 8 connection(s)
I20260809 19:28:01.339461 3065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:28:01.344430 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 3065
I20260809 19:28:01.344765 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.81.212:36843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:44147
--local_ip_for_outbound_sockets=127.21.81.195
--tserver_master_addrs=127.21.81.254:42829
--webserver_port=46421
--webserver_interface=127.21.81.195
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.81.212:36843/kms with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:28:01 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303681, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:01.351014 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:01.340349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:41903 (local address 127.21.81.254:42829)
0809 19:28:01.340406 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:01.340408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:01.340651 (+ 243us) server_negotiation.cc:407] Connection header received
0809 19:28:01.341134 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:01.341136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:01.341153 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:01.341256 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:01.341919 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:01.342854 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:01.343945 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:01.344303 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:01.344404 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:01.347428 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:01.347430 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:01.347430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:01.347438 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:01.349176 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:01.349800 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:01.349802 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:01.349802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:01.349867 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:01.350210 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:01.350211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:01.350211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:01.350367 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:01.350433 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:01.350824 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 19:28:01.350920 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:28:01.351238 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:01.340351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42829 (local address 127.21.81.194:41903)
0809 19:28:01.340536 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:01.340570 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:01.341019 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:01.341333 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:01.341337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:01.341566 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:01.341814 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:01.341819 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:01.342953 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:01.342954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:01.343838 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:01.343841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:01.344432 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:01.344434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:01.344582 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:01.345422 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:01.345446 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:01.347301 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:01.349272 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:01.349280 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:01.349282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:01.349697 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:01.349951 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:01.349953 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:01.349953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:01.350114 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:01.350444 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:01.350452 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:01.350641 (+ 189us) client_negotiation.cc:769] Sending connection context
0809 19:28:01.350756 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 19:28:01.350936 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
I20260809 19:28:01.351751 3203 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42829
I20260809 19:28:01.351843 3203 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:01.351989 3203 heartbeater.cc:511] Master 127.21.81.254:42829 requested a full tablet report, sending...
W20260809 19:28:01.352658 3203 heartbeater.cc:644] Failed to heartbeat to 127.21.81.254:42829 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 19:28:01.352715 3203 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:01.352758 3203 heartbeater.cc:511] Master 127.21.81.254:42829 requested a full tablet report, sending...
I20260809 19:28:01.353135 3203 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:01.353205 3203 heartbeater.cc:511] Master 127.21.81.254:42829 requested a full tablet report, sending...
W20260809 19:28:01.353535 3203 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:28:01.455783 3206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:01.456020 3206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:01.456060 3206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:01.457255 3206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:01.457340 3206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:01.457387 3206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:28:01.459015 3206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.81.212:36843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:44147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=46421
--tserver_master_addrs=127.21.81.254:42829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:01.459285 3206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:01.459522 3206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:01.465845 3215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:01.465919 3213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:01.465845 3211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:01.466711 3206 server_base.cc:1034] running on GCE node
I20260809 19:28:01.466928 3206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:01.467208 3206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:01.468364 3206 hybrid_clock.cc:648] HybridClock initialized: now 1786303681468209 us; error 167 us; skew 500 ppm
Aug 09 19:28:01 dist-test-slave-njxd krb5kdc[1794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303681, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:01.471277 3206 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:28:01.471719 3206 webserver.cc:460] Webserver started at http://127.21.81.195:46421/ using document root <none> and password file <none>
I20260809 19:28:01.471961 3206 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:01.472047 3206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 19:28:01 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303681, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
I20260809 19:28:01.494617 3206 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.000s
I20260809 19:28:01.495934 3224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:01.496151 3206 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:01.496254 3206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd"
format_stamp: "Formatted at 2026-08-09 19:28:00 on dist-test-slave-njxd"
server_key: "2ccb49a2415c0bed44622d4f6478cc79"
server_key_iv: "7d7d0c9886e1b83a74dd6d46b7c9f141"
server_key_version: "kuduclusterkey@0"
I20260809 19:28:01.496657 3206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:01.507632 3206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:01.510919 3206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:01.511154 3206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:01.511427 3206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:01.511821 3206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:01.511873 3206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:01.511904 3206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:01.511933 3206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:01.517953 3206 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:44147
I20260809 19:28:01.518348 3206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:28:01.524071 3347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:44147 every 8 connection(s)
I20260809 19:28:01.527944 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 3206
Aug 09 19:28:01 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303681, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:01.534343 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:01.524788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:50451 (local address 127.21.81.254:42829)
0809 19:28:01.524845 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:01.524847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:01.525094 (+ 247us) server_negotiation.cc:407] Connection header received
0809 19:28:01.525874 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:01.525875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:01.525888 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:01.525932 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:01.526522 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:01.527014 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:01.528041 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:01.528350 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:01.528419 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:01.531389 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:01.531391 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:01.531391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:01.531397 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:01.532860 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:01.533292 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:01.533293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:01.533294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:01.533351 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:01.533636 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:01.533637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:01.533637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:01.533748 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:01.533825 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:01.534173 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 19:28:01.534278 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:28:01.534932 3350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:01.524777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:42829 (local address 127.21.81.195:50451)
0809 19:28:01.524939 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:01.524991 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:01.525778 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:01.525998 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:01.526002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:01.526203 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:01.526407 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:01.526412 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:01.527111 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:01.527113 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:01.527946 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:01.527948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:01.528516 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:01.528517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:01.528663 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:01.529209 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:01.529223 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:01.531281 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:01.532963 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:01.532971 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:01.532973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:01.533205 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:01.533417 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:01.533418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:01.533418 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:01.533535 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:01.533823 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:01.533830 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:01.534005 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 19:28:01.534498 (+ 493us) client_negotiation.cc:240] Negotiation successful
0809 19:28:01.534702 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260809 19:28:01.535367 3348 heartbeater.cc:347] Connected to a master server at 127.21.81.254:42829
I20260809 19:28:01.535490 3348 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:01.535679 3348 heartbeater.cc:511] Master 127.21.81.254:42829 requested a full tablet report, sending...
W20260809 19:28:01.536329 3348 heartbeater.cc:644] Failed to heartbeat to 127.21.81.254:42829 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 19:28:01.536414 3348 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:01.536487 3348 heartbeater.cc:511] Master 127.21.81.254:42829 requested a full tablet report, sending...
I20260809 19:28:01.536855 3348 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:01.536926 3348 heartbeater.cc:511] Master 127.21.81.254:42829 requested a full tablet report, sending...
W20260809 19:28:01.537262 3348 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 19:28:01.537277 3352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:01.528355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47674 (local address 127.21.81.254:42829)
0809 19:28:01.528682 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:01.528684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:01.528690 (+ 6us) server_negotiation.cc:407] Connection header received
0809 19:28:01.528709 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:01.528710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:01.528721 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:01.528792 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:01.529517 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:01.530144 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:01.530998 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:01.531291 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:01.531355 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:01.533681 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:01.533682 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:01.533683 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:01.533690 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:01.535403 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:01.535824 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:01.535825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:01.535825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:01.535883 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:01.536253 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:01.536254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:01.536254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:01.536388 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:01.536718 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:01.536865 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 19:28:01.536957 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":59,"threads_started":1}
Aug 09 19:28:02 dist-test-slave-njxd krb5kdc[1794](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786303682, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:02.175414 3061 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:02.175673 3061 heartbeater.cc:511] Master 127.21.81.254:42829 requested a full tablet report, sending...
I20260809 19:28:02.176127 2850 ts_manager.cc:194] Registered new tserver with Master: a4d663b5cba841089b8caa1d9799db89 (127.21.81.193:41845)
I20260809 19:28:02.176841 2850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:39951
I20260809 19:28:02.353740 3203 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:02.353924 3203 heartbeater.cc:511] Master 127.21.81.254:42829 requested a full tablet report, sending...
I20260809 19:28:02.354331 2850 ts_manager.cc:194] Registered new tserver with Master: 422a92aefca74494815c5a64243df097 (127.21.81.194:33913)
I20260809 19:28:02.354913 2850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:41903
Aug 09 19:28:02 dist-test-slave-njxd krb5kdc[1794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303682, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for HTTP/127.21.81.212@KRBTEST.COM
I20260809 19:28:02.537443 3348 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:02.537670 3348 heartbeater.cc:511] Master 127.21.81.254:42829 requested a full tablet report, sending...
I20260809 19:28:02.538070 2850 ts_manager.cc:194] Registered new tserver with Master: af1cf1ead4de4bd2abcd6c8a9838a2fd (127.21.81.195:44147)
I20260809 19:28:02.538587 2850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:50451
I20260809 19:28:02.541391 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:28:02.557821 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:02.542545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47682 (local address 127.21.81.254:42829)
0809 19:28:02.542704 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:02.542707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:02.549914 (+ 7207us) server_negotiation.cc:407] Connection header received
0809 19:28:02.550029 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:02.550030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:02.550045 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:02.550101 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:02.550915 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.551633 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.552617 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.552885 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.552930 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:02.554556 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:02.554557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:02.554558 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:02.554583 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:02.556408 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:02.556838 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:02.556840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:02.556840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:02.556907 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:02.557129 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:02.557130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:02.557130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:02.557268 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:02.557350 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:02.557609 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 19:28:02.557713 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260809 19:28:02.560540 2850 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:28:02.562925 2850 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:28:02.574149 3376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:02.566207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:44147 (local address 127.0.0.1:41788)
0809 19:28:02.566436 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:02.566482 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:02.566553 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:02.569893 (+ 3340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:02.569897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:02.569900 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:02.570000 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:02.570003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.571587 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.571588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:02.572808 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:02.572812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.573781 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.573782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:02.573873 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:02.573907 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 19:28:02.573958 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:28:02.574019 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260809 19:28:02.574803 3380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:02.566627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33266 (local address 127.21.81.193:41845)
0809 19:28:02.566780 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:02.566786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:02.569897 (+ 3111us) server_negotiation.cc:407] Connection header received
0809 19:28:02.569942 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:02.569944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:02.569961 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:02.570036 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:02.570274 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.571351 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.572989 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.574036 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.574132 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:02.574198 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:02.574622 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 19:28:02.574687 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260809 19:28:02.575320 3379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:02.566549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41788 (local address 127.21.81.195:44147)
0809 19:28:02.569696 (+ 3147us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:02.569701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:02.569714 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:28:02.569751 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:02.569754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:02.569771 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:02.570100 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:02.570191 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.571503 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.572921 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.573682 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.574921 (+ 1239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:02.575167 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:02.575195 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 19:28:02.575241 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3108,"thread_start_us":85,"threads_started":1}
I20260809 19:28:02.574499 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:02.566434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:41845 (local address 127.0.0.1:33266)
0809 19:28:02.569755 (+ 3321us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:02.569767 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:02.569827 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:02.570148 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:02.570149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:02.570151 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:02.570194 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:02.570195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.571454 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.571455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:02.572839 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:02.572842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.574239 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.574240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:02.574300 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:02.574304 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:28:02.574358 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 19:28:02.574409 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3300,"mutex_wait_us":3,"thread_start_us":61,"threads_started":1}
I20260809 19:28:02.576017 3277 tablet_service.cc:1467] Processing CreateTablet for tablet 4ba523ee19f0492ca70cf76040c99918 (DEFAULT_TABLE table=test-table [id=e3c2f91e24db4004a5255e0635e6db9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:28:02.576176 2991 tablet_service.cc:1467] Processing CreateTablet for tablet 4ba523ee19f0492ca70cf76040c99918 (DEFAULT_TABLE table=test-table [id=e3c2f91e24db4004a5255e0635e6db9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:28:02.576447 2991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ba523ee19f0492ca70cf76040c99918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:02.576452 3277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ba523ee19f0492ca70cf76040c99918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:02.579514 3382 tablet_bootstrap.cc:492] T 4ba523ee19f0492ca70cf76040c99918 P a4d663b5cba841089b8caa1d9799db89: Bootstrap starting.
I20260809 19:28:02.580132 3382 tablet_bootstrap.cc:654] T 4ba523ee19f0492ca70cf76040c99918 P a4d663b5cba841089b8caa1d9799db89: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:02.580431 3382 log.cc:824] T 4ba523ee19f0492ca70cf76040c99918 P a4d663b5cba841089b8caa1d9799db89: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:02.581342 3382 tablet_bootstrap.cc:492] T 4ba523ee19f0492ca70cf76040c99918 P a4d663b5cba841089b8caa1d9799db89: No bootstrap required, opened a new log
I20260809 19:28:02.581439 3382 ts_tablet_manager.cc:1397] T 4ba523ee19f0492ca70cf76040c99918 P a4d663b5cba841089b8caa1d9799db89: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:28:02.584129 3382 raft_consensus.cc:348] T 4ba523ee19f0492ca70cf76040c99918 P a4d663b5cba841089b8caa1d9799db89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44147 } } peers { permanent_uuid: "a4d663b5cba841089b8caa1d9799db89" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41845 } } peers { permanent_uuid: "422a92aefca74494815c5a64243df097" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33913 } }
I20260809 19:28:02.584406 3382 raft_consensus.cc:374] T 4ba523ee19f0492ca70cf76040c99918 P a4d663b5cba841089b8caa1d9799db89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:02.584498 3382 raft_consensus.cc:729] T 4ba523ee19f0492ca70cf76040c99918 P a4d663b5cba841089b8caa1d9799db89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4d663b5cba841089b8caa1d9799db89, State: Initialized, Role: FOLLOWER
I20260809 19:28:02.584714 3382 consensus_queue.cc:260] T 4ba523ee19f0492ca70cf76040c99918 P a4d663b5cba841089b8caa1d9799db89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44147 } } peers { permanent_uuid: "a4d663b5cba841089b8caa1d9799db89" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41845 } } peers { permanent_uuid: "422a92aefca74494815c5a64243df097" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33913 } }
I20260809 19:28:02.585121 3382 ts_tablet_manager.cc:1428] T 4ba523ee19f0492ca70cf76040c99918 P a4d663b5cba841089b8caa1d9799db89: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260809 19:28:02.585170 3378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:02.566228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:33913 (local address 127.0.0.1:60070)
0809 19:28:02.566888 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:02.566899 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:02.566944 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:02.581163 (+ 14219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:02.581165 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:02.581168 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:02.581223 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:02.581225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.582536 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.582538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:02.583748 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:02.583751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.584887 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.584888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:02.584956 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:02.584961 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:28:02.585016 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 19:28:02.585065 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":97,"threads_started":1}
I20260809 19:28:02.585419 3381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:02.566752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60070 (local address 127.21.81.194:33913)
0809 19:28:02.574213 (+ 7461us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:02.574219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:02.574231 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:28:02.574268 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:02.574270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:02.574288 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:02.581169 (+ 6881us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:02.581302 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.582452 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.583881 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.584714 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.584787 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:02.584850 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:02.585295 (+ 445us) server_negotiation.cc:299] Negotiation successful
0809 19:28:02.585338 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7407,"thread_start_us":73,"threads_started":1}
I20260809 19:28:02.585434 3061 heartbeater.cc:503] Master 127.21.81.254:42829 was elected leader, sending a full tablet report...
I20260809 19:28:02.580861 3384 tablet_bootstrap.cc:492] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd: Bootstrap starting.
I20260809 19:28:02.586400 3125 tablet_service.cc:1467] Processing CreateTablet for tablet 4ba523ee19f0492ca70cf76040c99918 (DEFAULT_TABLE table=test-table [id=e3c2f91e24db4004a5255e0635e6db9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:28:02.586620 3384 tablet_bootstrap.cc:654] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:02.586786 3125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ba523ee19f0492ca70cf76040c99918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:02.587103 3384 log.cc:824] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:02.588443 3384 tablet_bootstrap.cc:492] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd: No bootstrap required, opened a new log
I20260809 19:28:02.588539 3384 ts_tablet_manager.cc:1397] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260809 19:28:02.590492 3387 tablet_bootstrap.cc:492] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097: Bootstrap starting.
I20260809 19:28:02.591233 3384 raft_consensus.cc:348] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44147 } } peers { permanent_uuid: "a4d663b5cba841089b8caa1d9799db89" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41845 } } peers { permanent_uuid: "422a92aefca74494815c5a64243df097" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33913 } }
I20260809 19:28:02.591445 3384 raft_consensus.cc:374] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:02.591531 3384 raft_consensus.cc:729] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af1cf1ead4de4bd2abcd6c8a9838a2fd, State: Initialized, Role: FOLLOWER
I20260809 19:28:02.591562 3387 tablet_bootstrap.cc:654] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:02.591698 3384 consensus_queue.cc:260] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44147 } } peers { permanent_uuid: "a4d663b5cba841089b8caa1d9799db89" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41845 } } peers { permanent_uuid: "422a92aefca74494815c5a64243df097" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33913 } }
I20260809 19:28:02.592065 3387 log.cc:824] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:02.592087 3384 ts_tablet_manager.cc:1428] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 19:28:02.592228 3348 heartbeater.cc:503] Master 127.21.81.254:42829 was elected leader, sending a full tablet report...
I20260809 19:28:02.592903 3387 tablet_bootstrap.cc:492] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097: No bootstrap required, opened a new log
I20260809 19:28:02.592972 3387 ts_tablet_manager.cc:1397] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:28:02.595356 3387 raft_consensus.cc:348] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44147 } } peers { permanent_uuid: "a4d663b5cba841089b8caa1d9799db89" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41845 } } peers { permanent_uuid: "422a92aefca74494815c5a64243df097" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33913 } }
I20260809 19:28:02.595510 3387 raft_consensus.cc:374] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:02.595597 3387 raft_consensus.cc:729] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 422a92aefca74494815c5a64243df097, State: Initialized, Role: FOLLOWER
I20260809 19:28:02.595724 3387 consensus_queue.cc:260] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44147 } } peers { permanent_uuid: "a4d663b5cba841089b8caa1d9799db89" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41845 } } peers { permanent_uuid: "422a92aefca74494815c5a64243df097" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33913 } }
I20260809 19:28:02.596030 3387 ts_tablet_manager.cc:1428] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:28:02.596071 3203 heartbeater.cc:503] Master 127.21.81.254:42829 was elected leader, sending a full tablet report...
I20260809 19:28:02.627247 3388 raft_consensus.cc:482] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:28:02.627419 3388 raft_consensus.cc:504] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44147 } } peers { permanent_uuid: "a4d663b5cba841089b8caa1d9799db89" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41845 } } peers { permanent_uuid: "422a92aefca74494815c5a64243df097" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33913 } }
I20260809 19:28:02.628070 3388 leader_election.cc:288] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4d663b5cba841089b8caa1d9799db89 (127.21.81.193:41845), 422a92aefca74494815c5a64243df097 (127.21.81.194:33913)
I20260809 19:28:02.632644 3381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:02.628132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:54341 (local address 127.21.81.194:33913)
0809 19:28:02.628180 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:02.628182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:02.628600 (+ 418us) server_negotiation.cc:407] Connection header received
0809 19:28:02.628703 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:02.628704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:02.628715 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:02.628745 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:02.629013 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.629747 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.631501 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.632109 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.632164 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:02.632173 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:02.632523 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 19:28:02.632569 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:28:02.632648 3380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:02.628322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:55909 (local address 127.21.81.193:41845)
0809 19:28:02.628379 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:02.628380 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:02.628440 (+ 60us) server_negotiation.cc:407] Connection header received
0809 19:28:02.628559 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:02.628560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:02.628572 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:02.628603 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:02.628961 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.629748 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.631501 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.632109 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.632164 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:02.632173 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:02.632523 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 19:28:02.632569 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 19:28:02.632926 3393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:02.628053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:41845 (local address 127.21.81.195:55909)
0809 19:28:02.628363 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:02.628377 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:02.628473 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:02.628706 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:02.628707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:02.628708 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:02.628849 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:02.628851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.629860 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.629861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:02.631385 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:02.631389 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.632301 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.632302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:02.632423 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:02.632426 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:28:02.632782 (+ 356us) client_negotiation.cc:240] Negotiation successful
0809 19:28:02.632840 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":102,"thread_start_us":122,"threads_started":1}
I20260809 19:28:02.633322 3012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ba523ee19f0492ca70cf76040c99918" candidate_uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4d663b5cba841089b8caa1d9799db89" is_pre_election: true
I20260809 19:28:02.633539 3012 raft_consensus.cc:2454] T 4ba523ee19f0492ca70cf76040c99918 P a4d663b5cba841089b8caa1d9799db89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af1cf1ead4de4bd2abcd6c8a9838a2fd in term 0.
I20260809 19:28:02.633912 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:02.628053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:33913 (local address 127.21.81.195:54341)
0809 19:28:02.628539 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:02.628550 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:02.628626 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:02.628836 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:02.628837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:02.628838 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:02.628930 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:02.628932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.629860 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.629861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:02.631384 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:02.631389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.632291 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.632293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:02.632421 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:02.632428 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 19:28:02.633106 (+ 678us) client_negotiation.cc:240] Negotiation successful
0809 19:28:02.633161 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":407,"thread_start_us":82,"threads_started":1}
I20260809 19:28:02.633896 3228 leader_election.cc:302] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4d663b5cba841089b8caa1d9799db89, af1cf1ead4de4bd2abcd6c8a9838a2fd; no voters:
I20260809 19:28:02.634198 3388 raft_consensus.cc:2781] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:28:02.634253 3388 raft_consensus.cc:482] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:28:02.634275 3388 raft_consensus.cc:3037] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:02.634366 3154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ba523ee19f0492ca70cf76040c99918" candidate_uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "422a92aefca74494815c5a64243df097" is_pre_election: true
I20260809 19:28:02.634521 3154 raft_consensus.cc:2454] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af1cf1ead4de4bd2abcd6c8a9838a2fd in term 0.
I20260809 19:28:02.635171 3388 raft_consensus.cc:504] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44147 } } peers { permanent_uuid: "a4d663b5cba841089b8caa1d9799db89" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41845 } } peers { permanent_uuid: "422a92aefca74494815c5a64243df097" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33913 } }
I20260809 19:28:02.635318 3388 leader_election.cc:288] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [CANDIDATE]: Term 1 election: Requested vote from peers a4d663b5cba841089b8caa1d9799db89 (127.21.81.193:41845), 422a92aefca74494815c5a64243df097 (127.21.81.194:33913)
I20260809 19:28:02.635478 3012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ba523ee19f0492ca70cf76040c99918" candidate_uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4d663b5cba841089b8caa1d9799db89"
I20260809 19:28:02.635505 3154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ba523ee19f0492ca70cf76040c99918" candidate_uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "422a92aefca74494815c5a64243df097"
I20260809 19:28:02.635551 3012 raft_consensus.cc:3037] T 4ba523ee19f0492ca70cf76040c99918 P a4d663b5cba841089b8caa1d9799db89 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:02.635604 3154 raft_consensus.cc:3037] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:02.636276 3012 raft_consensus.cc:2454] T 4ba523ee19f0492ca70cf76040c99918 P a4d663b5cba841089b8caa1d9799db89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af1cf1ead4de4bd2abcd6c8a9838a2fd in term 1.
I20260809 19:28:02.636473 3228 leader_election.cc:302] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4d663b5cba841089b8caa1d9799db89, af1cf1ead4de4bd2abcd6c8a9838a2fd; no voters:
I20260809 19:28:02.636587 3388 raft_consensus.cc:2781] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:28:02.636725 3154 raft_consensus.cc:2454] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af1cf1ead4de4bd2abcd6c8a9838a2fd in term 1.
I20260809 19:28:02.636776 3388 raft_consensus.cc:686] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [term 1 LEADER]: Becoming Leader. State: Replica: af1cf1ead4de4bd2abcd6c8a9838a2fd, State: Running, Role: LEADER
I20260809 19:28:02.636873 3388 consensus_queue.cc:237] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44147 } } peers { permanent_uuid: "a4d663b5cba841089b8caa1d9799db89" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41845 } } peers { permanent_uuid: "422a92aefca74494815c5a64243df097" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33913 } }
I20260809 19:28:02.637794 2850 catalog_manager.cc:5495] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd reported cstate change: term changed from 0 to 1, leader changed from <none> to af1cf1ead4de4bd2abcd6c8a9838a2fd (127.21.81.195). New cstate: current_term: 1 leader_uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af1cf1ead4de4bd2abcd6c8a9838a2fd" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4d663b5cba841089b8caa1d9799db89" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "422a92aefca74494815c5a64243df097" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33913 } health_report { overall_health: UNKNOWN } } }
I20260809 19:28:02.643505 3379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:02.640769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41804 (local address 127.21.81.195:44147)
0809 19:28:02.640810 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:02.640812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:02.640840 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:28:02.640918 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:02.640919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:02.640937 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:02.641010 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:28:02.641186 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.641812 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.642669 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:02.642907 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:02.642969 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:02.643162 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:28:02.643270 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:28:02.643357 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:02.643398 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 19:28:02.643420 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:28:02.645512 3012 raft_consensus.cc:1261] T 4ba523ee19f0492ca70cf76040c99918 P a4d663b5cba841089b8caa1d9799db89 [term 1 FOLLOWER]: Refusing update from remote peer af1cf1ead4de4bd2abcd6c8a9838a2fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:28:02.645524 3154 raft_consensus.cc:1261] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097 [term 1 FOLLOWER]: Refusing update from remote peer af1cf1ead4de4bd2abcd6c8a9838a2fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:28:02.645890 3388 consensus_queue.cc:1035] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4d663b5cba841089b8caa1d9799db89" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 41845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:28:02.646059 3395 consensus_queue.cc:1035] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "422a92aefca74494815c5a64243df097" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 19:28:02.648249 3062 tablet.cc:2369] T 4ba523ee19f0492ca70cf76040c99918 P a4d663b5cba841089b8caa1d9799db89: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:28:02.649394 3399 mvcc.cc:205] Tried to move back new op lower bound from 7316699884113379328 to 7316699884081192960. Current Snapshot: MvccSnapshot[applied={T|T < 7316699884113379328}]
I20260809 19:28:02.653182 2850 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:28:02.653275 2850 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:28:02.654062 2850 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd8c3573298444099d882f998e785f8e: Sending DeleteTablet for 3 replicas of tablet 4ba523ee19f0492ca70cf76040c99918
I20260809 19:28:02.654457 3125 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ba523ee19f0492ca70cf76040c99918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:02 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:60070
I20260809 19:28:02.654693 2991 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ba523ee19f0492ca70cf76040c99918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:02 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:33266
I20260809 19:28:02.654899 3277 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ba523ee19f0492ca70cf76040c99918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:02 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:41788
I20260809 19:28:02.655200 3411 tablet_replica.cc:330] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd: stopping tablet replica
I20260809 19:28:02.655318 3411 raft_consensus.cc:2229] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:28:02.655458 3411 raft_consensus.cc:2258] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:28:02.655532 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 2892
I20260809 19:28:02.660871 3411 ts_tablet_manager.cc:1905] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:28:02.662994 3411 ts_tablet_manager.cc:1918] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:28:02.663111 3411 log.cc:1196] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/4ba523ee19f0492ca70cf76040c99918
I20260809 19:28:02.663347 3409 tablet_replica.cc:330] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097: stopping tablet replica
I20260809 19:28:02.663481 3411 ts_tablet_manager.cc:1939] T 4ba523ee19f0492ca70cf76040c99918 P af1cf1ead4de4bd2abcd6c8a9838a2fd: Deleting consensus metadata
I20260809 19:28:02.663575 3409 raft_consensus.cc:2229] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:28:02.663746 3409 raft_consensus.cc:2258] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:28:02.664351 3409 ts_tablet_manager.cc:1905] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:28:02.664647 2835 catalog_manager.cc:4841] TS af1cf1ead4de4bd2abcd6c8a9838a2fd (127.21.81.195:44147): tablet 4ba523ee19f0492ca70cf76040c99918 (table test-table [id=e3c2f91e24db4004a5255e0635e6db9f]) successfully deleted
I20260809 19:28:02.668023 3409 ts_tablet_manager.cc:1918] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:28:02.668162 3409 log.cc:1196] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/4ba523ee19f0492ca70cf76040c99918
I20260809 19:28:02.668503 3409 ts_tablet_manager.cc:1939] T 4ba523ee19f0492ca70cf76040c99918 P 422a92aefca74494815c5a64243df097: Deleting consensus metadata
I20260809 19:28:02.668989 2833 catalog_manager.cc:4841] TS 422a92aefca74494815c5a64243df097 (127.21.81.194:33913): tablet 4ba523ee19f0492ca70cf76040c99918 (table test-table [id=e3c2f91e24db4004a5255e0635e6db9f]) successfully deleted
W20260809 19:28:02.678886 2835 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:41845 (error 108)
I20260809 19:28:02.679936 3378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:02.679760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:41845 (local address 127.0.0.1:33268)
0809 19:28:02.679847 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:02.679896 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:41845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 19:28:02.679930 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 3065
W20260809 19:28:02.680214 2835 catalog_manager.cc:4584] TS a4d663b5cba841089b8caa1d9799db89 (127.21.81.193:41845): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ba523ee19f0492ca70cf76040c99918: Network error: Client connection negotiation failed: client connection to 127.21.81.193:41845: connect: Connection refused (error 111)
I20260809 19:28:02.688413 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 3206
I20260809 19:28:02.696367 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 2815
I20260809 19:28:02.702982 21831 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 19:28:03.047186 21831 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 19:28:03.047274 21831 mini_ranger.cc:66] Stopping Ranger...
I20260809 19:28:03.401640 21831 mini_ranger.cc:68] Stopped Ranger
2026-08-09 19:28:03.401 UTC [1825] LOG: received smart shutdown request
2026-08-09 19:28:03.406 UTC [1825] LOG: background worker "logical replication launcher" (PID 1834) exited with exit code 1
2026-08-09 19:28:03.406 UTC [1829] LOG: shutting down
2026-08-09 19:28:03.573 UTC [1825] LOG: database system is shut down
2026-08-09T19:28:03Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48974 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:28:03 dist-test-slave-njxd krb5kdc[3423](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:28:03 dist-test-slave-njxd krb5kdc[3423](info): set up 2 sockets
Aug 09 19:28:03 dist-test-slave-njxd krb5kdc[3423](info): commencing operation
krb5kdc: starting...
W20260809 19:28:04.766022 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:28:04 dist-test-slave-njxd krb5kdc[3423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:28:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:28:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:28:04.887737 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:45251
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:45375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:45251,127.21.81.253:33137,127.21.81.252:33717
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:04.971076 3437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:04.971319 3437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:04.971354 3437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:04.972489 3437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:28:04.972555 3437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:04.972579 3437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:04.972613 3437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:28:04.972645 3437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:28:04.974224 3437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:45375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:45251,127.21.81.253:33137,127.21.81.252:33717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:45251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:04.974509 3437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:04.974768 3437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:04.980013 3445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:04.980013 3443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:04.980015 3442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:04.980119 3437 server_base.cc:1034] running on GCE node
I20260809 19:28:04.980562 3437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:04.980875 3437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:04.982057 3437 hybrid_clock.cc:648] HybridClock initialized: now 1786303684982033 us; error 51 us; skew 500 ppm
Aug 09 19:28:04 dist-test-slave-njxd krb5kdc[3423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303684, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:04.984486 3437 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:28:04.984907 3437 webserver.cc:460] Webserver started at http://127.21.81.254:44351/ using document root <none> and password file <none>
I20260809 19:28:04.985149 3437 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:04.985215 3437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:04.985385 3437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:28:04.986528 3437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "4e10d4667c894b03b623c0cc75b945bd"
format_stamp: "Formatted at 2026-08-09 19:28:04 on dist-test-slave-njxd"
server_key: "896653dd087da82133812f9f661f4118"
server_key_iv: "59f709da5a75481ca93f008bb84d474d"
server_key_version: "encryptionkey@0"
I20260809 19:28:04.986903 3437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "4e10d4667c894b03b623c0cc75b945bd"
format_stamp: "Formatted at 2026-08-09 19:28:04 on dist-test-slave-njxd"
server_key: "896653dd087da82133812f9f661f4118"
server_key_iv: "59f709da5a75481ca93f008bb84d474d"
server_key_version: "encryptionkey@0"
I20260809 19:28:04.988245 3437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:28:04.988958 3453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:04.989125 3437 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:28:04.989234 3437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "4e10d4667c894b03b623c0cc75b945bd"
format_stamp: "Formatted at 2026-08-09 19:28:04 on dist-test-slave-njxd"
server_key: "896653dd087da82133812f9f661f4118"
server_key_iv: "59f709da5a75481ca93f008bb84d474d"
server_key_version: "encryptionkey@0"
I20260809 19:28:04.989311 3437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:04.996807 3437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:04.998647 3437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:04.998813 3437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:05.002986 3437 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:45251
I20260809 19:28:05.003034 3512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:45251 every 8 connection(s)
I20260809 19:28:05.003386 3437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:28:05.004249 3513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:05.005901 3513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:28:05 dist-test-slave-njxd krb5kdc[3423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303684, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:05.012656 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 3437
I20260809 19:28:05.012782 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:28:05.012991 21831 external_mini_cluster.cc:1428] Setting key a34c79f72257820b19ab05b54c356b32
I20260809 19:28:05.015718 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.006617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54792 (local address 127.21.81.254:45251)
0809 19:28:05.006825 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:05.006835 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:05.006962 (+ 127us) server_negotiation.cc:407] Connection header received
0809 19:28:05.007518 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:05.007521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:05.007537 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:05.007630 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:05.008176 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.008989 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.009947 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.010273 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.010401 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.012330 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:05.012337 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:05.012337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:05.012344 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:05.014094 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.014497 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.014500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.014500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.014593 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.014876 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.014877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.014878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.015096 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:05.015170 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:05.015440 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 19:28:05.015542 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"spinlock_wait_cycles":8576,"thread_start_us":96,"threads_started":1}
I20260809 19:28:05.015827 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.006600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:45251 (local address 127.0.0.1:54792)
0809 19:28:05.006815 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.006858 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:05.007353 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:05.007643 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:05.007645 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:05.007922 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:05.008072 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.008076 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.009103 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.009104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.009850 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.009852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.010372 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.010373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.010450 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.010935 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:05.010943 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:05.012245 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:05.014202 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.014208 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.014211 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.014402 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.014667 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.014668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.014668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.014778 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.015173 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:05.015180 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:05.015265 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 19:28:05.015390 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 19:28:05.015661 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":86,"thread_start_us":108,"threads_started":1}
I20260809 19:28:05.016494 3513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:28:05.016858 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.016738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:33137 (local address 127.0.0.1:59156)
0809 19:28:05.016788 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.016835 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:33137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 19:28:05.016996 3454 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.81.253:33137: connect: Connection refused (error 111)
I20260809 19:28:05.017447 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.017322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:33137 (local address 127.0.0.1:59158)
0809 19:28:05.017373 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.017418 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:33137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 19:28:05.017606 3513 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.253:33137: Network error: Client connection negotiation failed: client connection to 127.21.81.253:33137: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.21.81.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.253.keytab.
Entry for principal kudu/127.21.81.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.253.keytab.
I20260809 19:28:05.040125 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.253:33137
--webserver_interface=127.21.81.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.253.keytab
--principal=kudu/127.21.81.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:45375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:45251,127.21.81.253:33137,127.21.81.252:33717
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:28:05.066886 3513 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } attempt: 1
I20260809 19:28:05.067461 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.067299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:33137 (local address 127.0.0.1:59174)
0809 19:28:05.067366 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.067415 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:33137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 19:28:05.067890 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.067774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:33137 (local address 127.0.0.1:59182)
0809 19:28:05.067822 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.067856 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:33137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 19:28:05.068087 3513 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.253:33137: Network error: Client connection negotiation failed: client connection to 127.21.81.253:33137: connect: Connection refused (error 111)
W20260809 19:28:05.125636 3520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:05.125855 3520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:05.125878 3520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:05.126974 3520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:28:05.127010 3520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:05.127025 3520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:05.127038 3520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:28:05.127050 3520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:28:05.128535 3520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:45375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:45251,127.21.81.253:33137,127.21.81.252:33717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.253.keytab
--principal=kudu/127.21.81.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.253:33137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.81.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:05.128737 3520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:05.128916 3520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:05.133808 3528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:05.133875 3526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:05.133876 3525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:05.134075 3520 server_base.cc:1034] running on GCE node
I20260809 19:28:05.134320 3520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:05.134662 3520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:05.135821 3520 hybrid_clock.cc:648] HybridClock initialized: now 1786303685135800 us; error 37 us; skew 500 ppm
I20260809 19:28:05.136269 3513 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } attempt: 2
I20260809 19:28:05.136687 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.136558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:33137 (local address 127.0.0.1:59184)
0809 19:28:05.136621 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.136657 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:33137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 19:28:05.137064 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.136976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:33137 (local address 127.0.0.1:59200)
0809 19:28:05.137024 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.137048 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:33137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 19:28:05.137178 3513 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.253:33137: Network error: Client connection negotiation failed: client connection to 127.21.81.253:33137: connect: Connection refused (error 111)
Aug 09 19:28:05 dist-test-slave-njxd krb5kdc[3423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303685, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:05.138664 3520 init.cc:377] Logged in from keytab as kudu/127.21.81.253@KRBTEST.COM (short username kudu)
I20260809 19:28:05.139076 3520 webserver.cc:460] Webserver started at http://127.21.81.253:38473/ using document root <none> and password file <none>
I20260809 19:28:05.139302 3520 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:05.139370 3520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:05.139519 3520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:28:05.140574 3520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/data/instance:
uuid: "eb6162d1c8104556a3aaae2a3a631b62"
format_stamp: "Formatted at 2026-08-09 19:28:05 on dist-test-slave-njxd"
server_key: "51c288224e3d91532eac5f9e8c85b446"
server_key_iv: "63d9049749f06bf863cf8cb6cec51cf9"
server_key_version: "encryptionkey@0"
I20260809 19:28:05.140934 3520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/wal/instance:
uuid: "eb6162d1c8104556a3aaae2a3a631b62"
format_stamp: "Formatted at 2026-08-09 19:28:05 on dist-test-slave-njxd"
server_key: "51c288224e3d91532eac5f9e8c85b446"
server_key_iv: "63d9049749f06bf863cf8cb6cec51cf9"
server_key_version: "encryptionkey@0"
I20260809 19:28:05.142391 3520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:28:05.143235 3536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:05.143443 3520 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:05.143554 3520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/wal
uuid: "eb6162d1c8104556a3aaae2a3a631b62"
format_stamp: "Formatted at 2026-08-09 19:28:05 on dist-test-slave-njxd"
server_key: "51c288224e3d91532eac5f9e8c85b446"
server_key_iv: "63d9049749f06bf863cf8cb6cec51cf9"
server_key_version: "encryptionkey@0"
I20260809 19:28:05.143651 3520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:05.161700 3520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:05.164194 3520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:05.164373 3520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:05.168644 3520 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.253:33137
I20260809 19:28:05.168684 3599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.253:33137 every 8 connection(s)
I20260809 19:28:05.169073 3520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/data/info.pb
I20260809 19:28:05.170018 3601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:05.171653 3601 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:28:05.175383 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 3520
I20260809 19:28:05.175518 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-1/wal/instance
I20260809 19:28:05.175732 21831 external_mini_cluster.cc:1428] Setting key 7be8a2086417bb79048675b4a6af9e6c
Aug 09 19:28:05 dist-test-slave-njxd krb5kdc[3423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303685, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.253@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:05.181121 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.172392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54806 (local address 127.21.81.254:45251)
0809 19:28:05.172461 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:05.172463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:05.172663 (+ 200us) server_negotiation.cc:407] Connection header received
0809 19:28:05.173104 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:05.173105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:05.173114 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:05.173177 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:05.173799 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.174302 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.175301 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.175544 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.175612 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.178253 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:05.178255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:05.178255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:05.178263 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:05.179683 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.180137 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.180138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.180139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.180203 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.180465 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.180467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.180467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.180594 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:05.180669 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:05.180938 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 19:28:05.181028 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:28:05.181332 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.172352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:45251 (local address 127.0.0.1:54806)
0809 19:28:05.172533 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.172576 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:05.173015 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:05.173258 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:05.173261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:05.173505 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:05.173718 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.173723 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.174384 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.174384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.175180 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.175182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.175646 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.175647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.175806 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.176484 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:05.176495 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:05.178150 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:05.179808 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.179815 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.179817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.180049 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.180273 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.180274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.180274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.180388 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.180660 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:05.180668 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:05.180808 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 19:28:05.180916 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 19:28:05.181080 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260809 19:28:05.181831 3601 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.81.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.252@KRBTEST.COM" created.
Aug 09 19:28:05 dist-test-slave-njxd krb5kdc[3423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303685, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.253@KRBTEST.COM for kudu/127.21.81.253@KRBTEST.COM
I20260809 19:28:05.190125 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.182100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:33137 (local address 127.0.0.1:59210)
0809 19:28:05.182153 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.182164 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:05.182206 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:05.182464 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:05.182465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:05.182611 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:05.182658 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.182660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.183516 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.183517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.184083 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.184085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.184557 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.184558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.184630 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.184999 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:05.185003 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:05.187095 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:05.189117 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.189118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.189118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.189278 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.189502 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.189502 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.189502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.189553 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.189914 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:05.189915 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:05.189951 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 19:28:05.190002 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:28:05.190066 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 19:28:05.190296 3605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.182173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59210 (local address 127.21.81.253:33137)
0809 19:28:05.182338 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:05.182341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:05.182357 (+ 16us) server_negotiation.cc:407] Connection header received
0809 19:28:05.182373 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:05.182374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:05.182384 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:05.182464 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:05.182743 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.183413 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.184172 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.184464 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.184558 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.187182 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:05.187189 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:05.187190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:05.187196 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:05.189030 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.189361 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.189364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.189365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.189431 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.189654 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.189655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.189655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.189845 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:05.189932 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:05.190050 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 19:28:05.190143 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260809 19:28:05.190845 3601 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:28:05.191190 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.191075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:33717 (local address 127.0.0.1:41852)
0809 19:28:05.191129 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.191172 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 19:28:05.191350 3537 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
I20260809 19:28:05.191735 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.191670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:33717 (local address 127.0.0.1:41858)
0809 19:28:05.191696 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.191719 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 19:28:05.191895 3601 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.252:33717: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.252.keytab.
Entry for principal kudu/127.21.81.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.252.keytab.
I20260809 19:28:05.201980 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.252:33717
--webserver_interface=127.21.81.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.252.keytab
--principal=kudu/127.21.81.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:45375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:45251,127.21.81.253:33137,127.21.81.252:33717
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:28:05.228335 3513 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } attempt: 3
Aug 09 19:28:05 dist-test-slave-njxd krb5kdc[3423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303684, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for kudu/127.21.81.253@KRBTEST.COM
I20260809 19:28:05.235623 3605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.228851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59216 (local address 127.21.81.253:33137)
0809 19:28:05.228929 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:05.228931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:05.228942 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:28:05.228973 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:05.228974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:05.228986 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:05.229060 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:05.229396 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.229998 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.230723 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.230942 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.231023 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.232931 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:05.232932 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:05.232933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:05.232940 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:05.234454 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.234805 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.234807 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.234807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.234866 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.235068 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.235069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.235069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.235205 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:05.235285 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:05.235450 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 19:28:05.235528 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 19:28:05.235623 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.228761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:33137 (local address 127.0.0.1:59216)
0809 19:28:05.228832 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.228842 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:05.228912 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:05.229071 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:05.229072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:05.229245 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:05.229309 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.229311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.230087 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.230089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.230631 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.230633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.231020 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.231021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.231109 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.231590 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:05.231595 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:05.232840 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:05.234541 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.234542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.234543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.234731 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.234941 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.234942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.234942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.235004 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.235276 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:05.235277 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:05.235323 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 19:28:05.235413 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 19:28:05.235515 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 19:28:05.236050 3513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:28:05.236415 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.236317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:33717 (local address 127.0.0.1:41874)
0809 19:28:05.236367 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.236398 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 19:28:05.236730 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.236641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:33717 (local address 127.0.0.1:41890)
0809 19:28:05.236685 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.236709 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 19:28:05.236824 3513 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.252:33717: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
I20260809 19:28:05.241159 3601 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } attempt: 1
I20260809 19:28:05.241539 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.241424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:33717 (local address 127.0.0.1:41892)
0809 19:28:05.241476 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.241504 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 19:28:05.241869 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.241775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:33717 (local address 127.0.0.1:41906)
0809 19:28:05.241822 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.241847 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 19:28:05.241964 3601 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.252:33717: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
I20260809 19:28:05.267968 3513 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } attempt: 1
I20260809 19:28:05.268450 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.268305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:33717 (local address 127.0.0.1:41922)
0809 19:28:05.268376 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.268416 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 19:28:05.268807 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.268711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:33717 (local address 127.0.0.1:41924)
0809 19:28:05.268757 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.268782 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 19:28:05.268895 3513 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.252:33717: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
W20260809 19:28:05.285055 3608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:05.285247 3608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:05.285269 3608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:05.286516 3608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:28:05.286556 3608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:05.286571 3608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:05.286583 3608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:28:05.286635 3608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:28:05.288105 3608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:45375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:45251,127.21.81.253:33137,127.21.81.252:33717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.252.keytab
--principal=kudu/127.21.81.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.252:33717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.81.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:05.288355 3608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:05.288591 3608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:05.293367 3615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:05.293385 3619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:05.293396 3616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:05.293676 3608 server_base.cc:1034] running on GCE node
I20260809 19:28:05.293983 3608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:05.294274 3608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:05.295439 3608 hybrid_clock.cc:648] HybridClock initialized: now 1786303685295424 us; error 26 us; skew 500 ppm
Aug 09 19:28:05 dist-test-slave-njxd krb5kdc[3423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303685, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:05.298004 3608 init.cc:377] Logged in from keytab as kudu/127.21.81.252@KRBTEST.COM (short username kudu)
I20260809 19:28:05.298480 3608 webserver.cc:460] Webserver started at http://127.21.81.252:43627/ using document root <none> and password file <none>
I20260809 19:28:05.298699 3608 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:05.298768 3608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:05.298924 3608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:28:05.299940 3608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/data/instance:
uuid: "e9d153e86dd24e57b2e142369efb5326"
format_stamp: "Formatted at 2026-08-09 19:28:05 on dist-test-slave-njxd"
server_key: "5298c017848ec21078c9cbf59a1e0f5b"
server_key_iv: "7a2f52b60ab4e9285b5af1f9941dbf96"
server_key_version: "encryptionkey@0"
I20260809 19:28:05.300312 3608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/wal/instance:
uuid: "e9d153e86dd24e57b2e142369efb5326"
format_stamp: "Formatted at 2026-08-09 19:28:05 on dist-test-slave-njxd"
server_key: "5298c017848ec21078c9cbf59a1e0f5b"
server_key_iv: "7a2f52b60ab4e9285b5af1f9941dbf96"
server_key_version: "encryptionkey@0"
I20260809 19:28:05.301688 3608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:28:05.302451 3626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:05.302620 3608 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:28:05.302724 3608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/wal
uuid: "e9d153e86dd24e57b2e142369efb5326"
format_stamp: "Formatted at 2026-08-09 19:28:05 on dist-test-slave-njxd"
server_key: "5298c017848ec21078c9cbf59a1e0f5b"
server_key_iv: "7a2f52b60ab4e9285b5af1f9941dbf96"
server_key_version: "encryptionkey@0"
I20260809 19:28:05.302810 3608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:05.310105 3601 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } attempt: 2
I20260809 19:28:05.310521 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.310392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:33717 (local address 127.0.0.1:41938)
0809 19:28:05.310454 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.310491 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 19:28:05.310881 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.310779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:33717 (local address 127.0.0.1:41942)
0809 19:28:05.310827 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.310860 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 19:28:05.310974 3601 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.252:33717: Network error: Client connection negotiation failed: client connection to 127.21.81.252:33717: connect: Connection refused (error 111)
I20260809 19:28:05.322654 3608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:05.325074 3608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:05.325258 3608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:05.329504 3608 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.252:33717
I20260809 19:28:05.329551 3680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.252:33717 every 8 connection(s)
I20260809 19:28:05.329957 3608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/data/info.pb
I20260809 19:28:05.330852 3681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:05.332419 3681 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:28:05.336716 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 3608
I20260809 19:28:05.336831 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303525464116-21831-0/minicluster-data/master-2/wal/instance
I20260809 19:28:05.337014 21831 external_mini_cluster.cc:1428] Setting key 78b2ea3daea4e83a52e3e1dfb0342571
Aug 09 19:28:05 dist-test-slave-njxd krb5kdc[3423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303685, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.252@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:05.341931 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.333151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54810 (local address 127.21.81.254:45251)
0809 19:28:05.333232 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:05.333234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:05.333438 (+ 204us) server_negotiation.cc:407] Connection header received
0809 19:28:05.333947 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:05.333949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:05.333959 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:05.334011 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:05.334642 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.335109 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.336053 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.336241 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.336291 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.338961 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:05.338962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:05.338963 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:05.338971 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:05.340485 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.340853 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.340854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.340855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.340923 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.341227 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.341228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.341228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.341377 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:05.341444 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:05.341797 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 19:28:05.341867 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 19:28:05.342207 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.333100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:45251 (local address 127.0.0.1:54810)
0809 19:28:05.333309 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.333345 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:05.333860 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:05.334081 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:05.334086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:05.334330 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:05.334541 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.334545 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.335184 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.335185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.335982 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.335984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.336313 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.336313 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.336411 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.336976 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:05.336984 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:05.338869 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:05.340561 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.340567 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.340569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.340780 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.341006 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.341007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.341008 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.341152 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.341449 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:05.341457 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:05.341648 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 19:28:05.341773 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 19:28:05.341942 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":112,"threads_started":1}
I20260809 19:28:05.342801 3681 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:28:05 dist-test-slave-njxd krb5kdc[3423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.252@KRBTEST.COM
I20260809 19:28:05.344029 3513 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } attempt: 2
I20260809 19:28:05.346941 3685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.337433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41946 (local address 127.21.81.252:33717)
0809 19:28:05.337625 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:05.337629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:05.337674 (+ 45us) server_negotiation.cc:407] Connection header received
0809 19:28:05.337742 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:05.337745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:05.337771 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:05.337839 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:05.338583 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.339560 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.340413 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.340795 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.340887 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.343389 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:05.343397 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:05.343398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:05.343404 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:05.345131 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.345532 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.345535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.345536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.345633 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.345836 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.345837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.345837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.346025 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:05.346206 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:05.346750 (+ 544us) server_negotiation.cc:299] Negotiation successful
0809 19:28:05.346829 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
Aug 09 19:28:05 dist-test-slave-njxd krb5kdc[3423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303685, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.252@KRBTEST.COM for kudu/127.21.81.253@KRBTEST.COM
Aug 09 19:28:05 dist-test-slave-njxd krb5kdc[3423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303684, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for kudu/127.21.81.252@KRBTEST.COM
I20260809 19:28:05.350395 3605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.343157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59232 (local address 127.21.81.253:33137)
0809 19:28:05.343204 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:05.343206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:05.343214 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:28:05.343237 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:05.343240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:05.343252 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:05.343314 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:05.343594 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.344189 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.345256 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.345468 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.345551 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.347558 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:05.347560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:05.347560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:05.347567 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:05.349165 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.349614 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.349616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.349616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.349676 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.349868 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.349869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.349869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.349985 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:05.350061 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:05.350219 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 19:28:05.350297 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:28:05.350407 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.343051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:33137 (local address 127.0.0.1:59232)
0809 19:28:05.343110 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.343123 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:05.343164 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:05.343318 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:05.343319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:05.343446 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:05.343489 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.343490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.344307 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.344308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.345152 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.345154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.345635 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.345636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.345717 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.346092 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:05.346097 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:05.347476 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:05.349283 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.349285 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.349285 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.349507 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.349753 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.349754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.349754 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.349800 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.350055 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:05.350056 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:05.350097 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 19:28:05.350181 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 19:28:05.350317 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":9}
I20260809 19:28:05.350865 3681 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:28:05.352694 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.344295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:33717 (local address 127.0.0.1:41958)
0809 19:28:05.344494 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.344507 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:05.344556 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:05.344701 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:05.344701 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:05.344852 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:05.344885 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.344886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.345779 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.345780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.346540 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.346543 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.347095 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.347096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.347151 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.347570 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:05.347576 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:05.348953 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:05.351597 (+ 2644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.351598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.351599 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.351819 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.352026 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.352027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.352027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.352092 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.352369 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:05.352370 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:05.352419 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 19:28:05.352497 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 19:28:05.352603 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180}
I20260809 19:28:05.352751 3686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.344332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41958 (local address 127.21.81.252:33717)
0809 19:28:05.344497 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:05.344500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:05.344540 (+ 40us) server_negotiation.cc:407] Connection header received
0809 19:28:05.344621 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:05.344622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:05.344631 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:05.344690 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:05.344963 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.345681 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.346655 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.346991 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.347285 (+ 294us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.349014 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:05.349016 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:05.349016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:05.349023 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:05.351515 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.351892 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.351893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.351893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.351956 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.352166 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.352167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.352168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.352279 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:05.352369 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:05.352536 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 19:28:05.352612 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":117,"threads_started":1}
Aug 09 19:28:05 dist-test-slave-njxd krb5kdc[3423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
Aug 09 19:28:05 dist-test-slave-njxd krb5kdc[3423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303685, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.252@KRBTEST.COM for kudu/127.21.81.252@KRBTEST.COM
I20260809 19:28:05.355242 3513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd: Bootstrap starting.
I20260809 19:28:05.356077 3513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:05.356431 3513 log.cc:824] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:05.357234 3513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd: No bootstrap required, opened a new log
I20260809 19:28:05.357986 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.348871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54818 (local address 127.21.81.254:45251)
0809 19:28:05.348907 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:05.348908 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:05.348941 (+ 33us) server_negotiation.cc:407] Connection header received
0809 19:28:05.348962 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:05.348963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:05.348973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:05.349017 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:05.349653 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.350397 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.351754 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.352039 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.352111 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.354957 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:05.354959 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:05.354959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:05.354966 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:05.356684 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.357088 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.357089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.357089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.357143 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.357357 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.357358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.357359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.357483 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:05.357621 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:05.357829 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 19:28:05.357897 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:28:05.358141 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.351118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:33717 (local address 127.0.0.1:41964)
0809 19:28:05.351152 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.351161 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:05.351206 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:05.351326 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:05.351327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:05.351507 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:05.351530 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.351531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.352385 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.352386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.353016 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.353018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.353404 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.353405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.353450 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.353855 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:05.353860 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:05.355350 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:05.357121 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.357122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.357123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.357331 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.357538 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.357539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.357539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.357583 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.357883 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:05.357883 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:05.357917 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 19:28:05.357969 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 19:28:05.358053 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 19:28:05.358440 3685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.351163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41964 (local address 127.21.81.252:33717)
0809 19:28:05.351197 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:05.351199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:05.351207 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:28:05.351262 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:05.351263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:05.351272 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:05.351323 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:05.351623 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.352309 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.353111 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.353323 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.353402 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.355434 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:05.355435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:05.355436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:05.355443 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:05.357039 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.357419 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.357432 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.357433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.357481 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.357661 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.357662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.357662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.357804 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:05.357863 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:05.358294 (+ 431us) server_negotiation.cc:299] Negotiation successful
0809 19:28:05.358369 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:28:05.359566 3513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } }
I20260809 19:28:05.359701 3513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:05.359766 3513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e10d4667c894b03b623c0cc75b945bd, State: Initialized, Role: FOLLOWER
I20260809 19:28:05.359851 3681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326: Bootstrap starting.
I20260809 19:28:05.359895 3513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } }
I20260809 19:28:05.360176 3688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } } }
I20260809 19:28:05.360248 3513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:28:05.360324 3688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:28:05.360574 3681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:05.360893 3681 log.cc:824] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:05.361743 3681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326: No bootstrap required, opened a new log
W20260809 19:28:05.362285 3701 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:28:05.362334 3701 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 19:28:05 dist-test-slave-njxd krb5kdc[3423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.253@KRBTEST.COM
I20260809 19:28:05.363955 3681 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } }
I20260809 19:28:05.364070 3681 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:05.364092 3681 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d153e86dd24e57b2e142369efb5326, State: Initialized, Role: FOLLOWER
I20260809 19:28:05.364204 3681 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } }
I20260809 19:28:05.364449 3703 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } } }
I20260809 19:28:05.364504 3681 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:28:05.364513 3703 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:28:05.363310 3437 master_runner.cc:186] Error getting master registration for 127.21.81.253:33137: OK, instance_id { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" instance_seqno: 1786303685167434 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260809 19:28:05.366596 3714 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 19:28:05.366734 3437 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 19:28:05.366950 3714 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:28:05.367689 3605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.358627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59248 (local address 127.21.81.253:33137)
0809 19:28:05.358663 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:05.358665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:05.358766 (+ 101us) server_negotiation.cc:407] Connection header received
0809 19:28:05.358825 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:05.358826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:05.358836 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:05.358887 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:05.359492 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.360177 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.361155 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.361479 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.361553 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.364357 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:05.364358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:05.364358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:05.364378 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:05.366109 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.366536 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.366537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.366538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.366597 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.366826 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.366827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.366828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.366956 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:05.367147 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:05.367514 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 19:28:05.367596 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:28:05.369364 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:28:05.368148 3608 master_runner.cc:186] Error getting master registration for 127.21.81.253:33137: OK, instance_id { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" instance_seqno: 1786303685167434 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:28:05.370396 3608 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:28:05.377101 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.370933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54830 (local address 127.21.81.254:45251)
0809 19:28:05.370979 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:05.370981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:05.371113 (+ 132us) server_negotiation.cc:407] Connection header received
0809 19:28:05.371161 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:05.371162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:05.371172 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:05.371234 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:05.371775 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.372234 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.372961 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.373152 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.373229 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.374478 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:05.374479 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:05.374479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:05.374486 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:05.375878 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.376267 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.376269 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.376269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.376327 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.376609 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.376610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.376610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.376710 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:05.376816 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:05.376916 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 19:28:05.376992 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:28:05.384075 3605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.377842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59258 (local address 127.21.81.253:33137)
0809 19:28:05.377892 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:05.377894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:05.378057 (+ 163us) server_negotiation.cc:407] Connection header received
0809 19:28:05.378118 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:05.378119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:05.378130 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:05.378221 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:05.378691 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.379144 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.379806 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.379998 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.380081 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.381230 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:05.381232 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:05.381232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:05.381239 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:05.382761 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.383216 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.383217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.383218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.383281 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.383505 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.383506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.383506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.383626 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:05.383712 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:05.383906 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 19:28:05.383982 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 19:28:05.389201 3685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.377949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41966 (local address 127.21.81.252:33717)
0809 19:28:05.377989 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:05.377991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:05.382054 (+ 4063us) server_negotiation.cc:407] Connection header received
0809 19:28:05.382212 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:05.382213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:05.382233 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:05.382259 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:05.383513 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.384170 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.385166 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.385361 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.385447 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.386610 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:05.386611 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:05.386612 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:05.386619 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:05.388009 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.388369 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.388370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.388371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.388430 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.388634 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.388635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.388636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.388754 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:05.388856 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:05.389008 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 19:28:05.389087 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:28:05.402101 3601 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } attempt: 3
Aug 09 19:28:05 dist-test-slave-njxd krb5kdc[3423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303685, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.253@KRBTEST.COM for kudu/127.21.81.252@KRBTEST.COM
I20260809 19:28:05.410017 3685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.402771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41974 (local address 127.21.81.252:33717)
0809 19:28:05.402810 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:05.402812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:05.402848 (+ 36us) server_negotiation.cc:407] Connection header received
0809 19:28:05.402922 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:05.402923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:05.402933 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:05.402986 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:05.403350 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.403819 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.404531 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.404779 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.404885 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.407424 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:05.407425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:05.407426 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:05.407433 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:05.408827 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.409178 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.409179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.409180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.409249 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:05.409456 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:05.409457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:05.409458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:05.409569 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:05.409668 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:05.409845 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 19:28:05.409917 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:28:05.410017 3603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:05.402747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:33717 (local address 127.0.0.1:41974)
0809 19:28:05.402806 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:05.402817 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:05.402864 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:05.403053 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:05.403054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:05.403223 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:05.403267 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.403269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.403908 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.403909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.404440 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:05.404441 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:05.404887 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:05.404889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:05.404956 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:05.406086 (+ 1130us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:05.406092 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:05.407333 (+ 1241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:05.408930 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.408931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.408931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.409105 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.409324 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:05.409325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:05.409325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:05.409387 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:05.409674 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:05.409675 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:05.409728 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 19:28:05.409806 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 19:28:05.409911 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 19:28:05.411909 3601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62: Bootstrap starting.
I20260809 19:28:05.412667 3601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:05.412988 3601 log.cc:824] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:05.414008 3601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62: No bootstrap required, opened a new log
I20260809 19:28:05.415678 3601 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } }
I20260809 19:28:05.415815 3601 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:05.415880 3601 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6162d1c8104556a3aaae2a3a631b62, State: Initialized, Role: FOLLOWER
I20260809 19:28:05.416038 3601 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } }
I20260809 19:28:05.416330 3725 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } } }
I20260809 19:28:05.416383 3601 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:28:05.416465 3725 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:28:05.418360 3736 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:28:05.418449 3736 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:28:05.420214 3520 master_runner.cc:263] No leader master found from master eb6162d1c8104556a3aaae2a3a631b62
I20260809 19:28:05.420287 3520 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 19:28:05.499588 3460 tablet.cc:2369] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:28:05.570036 3703 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:28:05.570169 3703 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } }
I20260809 19:28:05.570402 3703 leader_election.cc:288] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e10d4667c894b03b623c0cc75b945bd (127.21.81.254:45251), eb6162d1c8104556a3aaae2a3a631b62 (127.21.81.253:33137)
I20260809 19:28:05.570653 3483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e9d153e86dd24e57b2e142369efb5326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e10d4667c894b03b623c0cc75b945bd" is_pre_election: true
I20260809 19:28:05.570732 3563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e9d153e86dd24e57b2e142369efb5326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6162d1c8104556a3aaae2a3a631b62" is_pre_election: true
I20260809 19:28:05.570863 3483 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9d153e86dd24e57b2e142369efb5326 in term 0.
I20260809 19:28:05.570863 3563 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9d153e86dd24e57b2e142369efb5326 in term 0.
I20260809 19:28:05.571075 3628 leader_election.cc:302] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e10d4667c894b03b623c0cc75b945bd, e9d153e86dd24e57b2e142369efb5326; no voters:
I20260809 19:28:05.571223 3703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:28:05.571298 3703 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:28:05.571316 3703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:05.572067 3703 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } }
I20260809 19:28:05.572216 3703 leader_election.cc:288] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [CANDIDATE]: Term 1 election: Requested vote from peers 4e10d4667c894b03b623c0cc75b945bd (127.21.81.254:45251), eb6162d1c8104556a3aaae2a3a631b62 (127.21.81.253:33137)
I20260809 19:28:05.572371 3483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e9d153e86dd24e57b2e142369efb5326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e10d4667c894b03b623c0cc75b945bd"
I20260809 19:28:05.572402 3563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e9d153e86dd24e57b2e142369efb5326" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6162d1c8104556a3aaae2a3a631b62"
I20260809 19:28:05.572468 3483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:05.572476 3563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:05.573169 3483 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9d153e86dd24e57b2e142369efb5326 in term 1.
I20260809 19:28:05.573238 3563 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9d153e86dd24e57b2e142369efb5326 in term 1.
I20260809 19:28:05.573382 3628 leader_election.cc:302] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e10d4667c894b03b623c0cc75b945bd, e9d153e86dd24e57b2e142369efb5326; no voters:
I20260809 19:28:05.573499 3703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:28:05.573724 3703 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [term 1 LEADER]: Becoming Leader. State: Replica: e9d153e86dd24e57b2e142369efb5326, State: Running, Role: LEADER
I20260809 19:28:05.573854 3703 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } }
I20260809 19:28:05.574302 3738 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9d153e86dd24e57b2e142369efb5326. Latest consensus state: current_term: 1 leader_uuid: "e9d153e86dd24e57b2e142369efb5326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } } }
I20260809 19:28:05.574398 3738 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:05.574635 3740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:28:05.574826 3740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260809 19:28:05.575939 3632 tablet.cc:2369] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:28:05.576187 3483 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd [term 1 FOLLOWER]: Refusing update from remote peer e9d153e86dd24e57b2e142369efb5326: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:28:05.576315 3563 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62 [term 1 FOLLOWER]: Refusing update from remote peer e9d153e86dd24e57b2e142369efb5326: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:28:05.576617 3738 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:28:05.576766 3703 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:28:05.577970 3725 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9d153e86dd24e57b2e142369efb5326. Latest consensus state: current_term: 1 leader_uuid: "e9d153e86dd24e57b2e142369efb5326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } } }
I20260809 19:28:05.577975 3688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9d153e86dd24e57b2e142369efb5326. Latest consensus state: current_term: 1 leader_uuid: "e9d153e86dd24e57b2e142369efb5326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } } }
I20260809 19:28:05.578060 3725 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:28:05.578080 3688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:28:05.578998 3703 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9d153e86dd24e57b2e142369efb5326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } } }
I20260809 19:28:05.579077 3703 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:05.579408 3741 mvcc.cc:205] Tried to move back new op lower bound from 7316699896117719040 to 7316699896111222784. Current Snapshot: MvccSnapshot[applied={T|T < 7316699896117719040 or (T in {7316699896117719040})}]
I20260809 19:28:05.579476 3688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9d153e86dd24e57b2e142369efb5326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } } }
I20260809 19:28:05.579488 3703 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9d153e86dd24e57b2e142369efb5326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } } }
I20260809 19:28:05.579556 3688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:28:05.579572 3703 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326 [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:05.579684 3740 catalog_manager.cc:1347] Generated new cluster ID: 4c65ae9455754d61bf924f46f9f315a3
I20260809 19:28:05.579720 3740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:28:05.579896 3725 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9d153e86dd24e57b2e142369efb5326" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e10d4667c894b03b623c0cc75b945bd" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45251 } } peers { permanent_uuid: "eb6162d1c8104556a3aaae2a3a631b62" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 33137 } } peers { permanent_uuid: "e9d153e86dd24e57b2e142369efb5326" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 33717 } } }
I20260809 19:28:05.579968 3725 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:28:05.580508 3742 mvcc.cc:205] Tried to move back new op lower bound from 7316699896117719040 to 7316699896111222784. Current Snapshot: MvccSnapshot[applied={T|T < 7316699896117719040}]
I20260809 19:28:05.594764 3740 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:28:05.595295 3740 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:28:05.604084 3740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9d153e86dd24e57b2e142369efb5326: Generated new TSK 0
I20260809 19:28:05.604319 3740 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:28:06.362775 3701 catalog_manager.cc:1259] Loaded cluster ID: 4c65ae9455754d61bf924f46f9f315a3
I20260809 19:28:06.362857 3701 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd: loading cluster ID for follower catalog manager: success
I20260809 19:28:06.363651 3701 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd: acquiring CA information for follower catalog manager: success
I20260809 19:28:06.363905 3701 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4e10d4667c894b03b623c0cc75b945bd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:28:06.418900 3736 catalog_manager.cc:1259] Loaded cluster ID: 4c65ae9455754d61bf924f46f9f315a3
I20260809 19:28:06.418980 3736 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62: loading cluster ID for follower catalog manager: success
I20260809 19:28:06.419818 3736 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62: acquiring CA information for follower catalog manager: success
I20260809 19:28:06.420140 3736 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eb6162d1c8104556a3aaae2a3a631b62: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:28:06.672796 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 3437
I20260809 19:28:06.680910 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 3520
I20260809 19:28:06.687755 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 3608
2026-08-09T19:28:06Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2992 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:28:06 dist-test-slave-njxd krb5kdc[3757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:28:06 dist-test-slave-njxd krb5kdc[3757](info): set up 2 sockets
Aug 09 19:28:06 dist-test-slave-njxd krb5kdc[3757](info): commencing operation
krb5kdc: starting...
W20260809 19:28:07.758924 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:28:07 dist-test-slave-njxd krb5kdc[3757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:28:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:28:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:28:07.882398 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:40051
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:41051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:40051
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:07.963451 3771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:07.963770 3771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:07.963825 3771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:07.965015 3771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:28:07.965090 3771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:07.965114 3771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:07.965137 3771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:28:07.965165 3771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:28:07.966650 3771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:41051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:40051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:40051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:07.966929 3771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:07.967154 3771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:07.972034 3777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:07.972041 3778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:07.972278 3771 server_base.cc:1034] running on GCE node
W20260809 19:28:07.972280 3780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:07.972558 3771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:07.972851 3771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:07.974022 3771 hybrid_clock.cc:648] HybridClock initialized: now 1786303687974016 us; error 36 us; skew 500 ppm
Aug 09 19:28:07 dist-test-slave-njxd krb5kdc[3757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303687, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:07.976542 3771 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:28:07.976994 3771 webserver.cc:460] Webserver started at http://127.21.81.254:44265/ using document root <none> and password file <none>
I20260809 19:28:07.977171 3771 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:07.977218 3771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:07.977308 3771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:28:07.978364 3771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "27717b7c74b644fdb61ba57abe41d4ba"
format_stamp: "Formatted at 2026-08-09 19:28:07 on dist-test-slave-njxd"
server_key: "94ac4ac5432d137112fa09bfe21b13cc"
server_key_iv: "0e957555015c84c2a20be5e2d0569a14"
server_key_version: "encryptionkey@0"
I20260809 19:28:07.978664 3771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "27717b7c74b644fdb61ba57abe41d4ba"
format_stamp: "Formatted at 2026-08-09 19:28:07 on dist-test-slave-njxd"
server_key: "94ac4ac5432d137112fa09bfe21b13cc"
server_key_iv: "0e957555015c84c2a20be5e2d0569a14"
server_key_version: "encryptionkey@0"
I20260809 19:28:07.979931 3771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:28:07.980612 3787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:07.980794 3771 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:28:07.980890 3771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "27717b7c74b644fdb61ba57abe41d4ba"
format_stamp: "Formatted at 2026-08-09 19:28:07 on dist-test-slave-njxd"
server_key: "94ac4ac5432d137112fa09bfe21b13cc"
server_key_iv: "0e957555015c84c2a20be5e2d0569a14"
server_key_version: "encryptionkey@0"
I20260809 19:28:07.980945 3771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:07.992823 3771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:07.995123 3771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:07.995244 3771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:07.999228 3771 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:40051
I20260809 19:28:07.999300 3842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:40051 every 8 connection(s)
I20260809 19:28:07.999648 3771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:28:08.000528 3843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:08.002919 3843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba: Bootstrap starting.
I20260809 19:28:08.003515 3843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:08.003823 3843 log.cc:824] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:08.004510 3843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba: No bootstrap required, opened a new log
I20260809 19:28:08.006067 3843 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27717b7c74b644fdb61ba57abe41d4ba" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40051 } }
I20260809 19:28:08.006222 3843 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:08.006270 3843 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27717b7c74b644fdb61ba57abe41d4ba, State: Initialized, Role: FOLLOWER
I20260809 19:28:08.006429 3843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27717b7c74b644fdb61ba57abe41d4ba" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40051 } }
I20260809 19:28:08.006527 3843 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:28:08.006570 3843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:28:08.006620 3843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:08.006940 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 3771
I20260809 19:28:08.007019 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:28:08.007189 3843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27717b7c74b644fdb61ba57abe41d4ba" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40051 } }
I20260809 19:28:08.007216 21831 external_mini_cluster.cc:1428] Setting key be8660ef6907395b38d02395c83139e6
I20260809 19:28:08.007305 3843 leader_election.cc:302] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27717b7c74b644fdb61ba57abe41d4ba; no voters:
I20260809 19:28:08.007491 3843 leader_election.cc:288] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:28:08.007510 3847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:28:08.007634 3847 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [term 1 LEADER]: Becoming Leader. State: Replica: 27717b7c74b644fdb61ba57abe41d4ba, State: Running, Role: LEADER
I20260809 19:28:08.007761 3843 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:28:08.007737 3847 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27717b7c74b644fdb61ba57abe41d4ba" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40051 } }
I20260809 19:28:08.008234 3848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27717b7c74b644fdb61ba57abe41d4ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27717b7c74b644fdb61ba57abe41d4ba" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40051 } } }
I20260809 19:28:08.008388 3849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27717b7c74b644fdb61ba57abe41d4ba. Latest consensus state: current_term: 1 leader_uuid: "27717b7c74b644fdb61ba57abe41d4ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27717b7c74b644fdb61ba57abe41d4ba" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40051 } } }
I20260809 19:28:08.008476 3849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:08.008386 3848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:08.008671 3859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:28:08.009529 3859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:28:08.011333 3859 catalog_manager.cc:1347] Generated new cluster ID: bab38d66e5374a5598f59e5601884cb2
I20260809 19:28:08.011389 3859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:28:08 dist-test-slave-njxd krb5kdc[3757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:08.019907 3851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:08.007796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59176 (local address 127.21.81.254:40051)
0809 19:28:08.008400 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:08.008409 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:08.008443 (+ 34us) server_negotiation.cc:407] Connection header received
0809 19:28:08.008559 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:08.008569 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:08.008627 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:08.009741 (+ 1114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:08.009934 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:08.011177 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:08.012213 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:08.012850 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:08.012977 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:08.015840 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:08.015844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:08.015845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:08.015857 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:08.018241 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:08.018660 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:08.018663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:08.018664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:08.018778 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:08.018994 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:08.018995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:08.018995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:08.019280 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:08.019422 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:08.019545 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 19:28:08.019677 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":72,"threads_started":1}
I20260809 19:28:08.020848 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:28:08.027741 3859 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:28:08.028394 3859 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:28:08.033195 3859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27717b7c74b644fdb61ba57abe41d4ba: Generated new TSK 0
I20260809 19:28:08.033372 3859 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:28:08.074854 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 3771
2026-08-09T19:28:08Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1386 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:28:08 dist-test-slave-njxd krb5kdc[3872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:28:08 dist-test-slave-njxd krb5kdc[3872](info): set up 2 sockets
Aug 09 19:28:08 dist-test-slave-njxd krb5kdc[3872](info): commencing operation
krb5kdc: starting...
W20260809 19:28:09.141252 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:28:09 dist-test-slave-njxd krb5kdc[3872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:28:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:28:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:28:09.261049 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:33815
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:33815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:09.343297 3886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:09.343505 3886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:09.343528 3886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:09.344616 3886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:28:09.344648 3886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:09.344662 3886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:09.344676 3886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:28:09.344687 3886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:28:09.346184 3886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:33815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:33815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:09.346390 3886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:09.346555 3886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:09.351272 3892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:09.351322 3891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:09.351239 3894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:09.351454 3886 server_base.cc:1034] running on GCE node
I20260809 19:28:09.351727 3886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:09.352058 3886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:09.353209 3886 hybrid_clock.cc:648] HybridClock initialized: now 1786303689353189 us; error 39 us; skew 500 ppm
Aug 09 19:28:09 dist-test-slave-njxd krb5kdc[3872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303689, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:09.355805 3886 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:28:09.356206 3886 webserver.cc:460] Webserver started at http://127.21.81.254:33175/ using document root <none> and password file <none>
I20260809 19:28:09.356403 3886 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:09.356444 3886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:09.356531 3886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:28:09.357530 3886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "409dc59891924d03b336f94aa280d3f2"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "6b33480b91a610796dbe1ff76542641a"
server_key_iv: "9c78db8cf574888ef8a705b153610a8c"
server_key_version: "encryptionkey@0"
I20260809 19:28:09.357939 3886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "409dc59891924d03b336f94aa280d3f2"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "6b33480b91a610796dbe1ff76542641a"
server_key_iv: "9c78db8cf574888ef8a705b153610a8c"
server_key_version: "encryptionkey@0"
I20260809 19:28:09.359234 3886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:28:09.359962 3902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:09.360131 3886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:28:09.360244 3886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "409dc59891924d03b336f94aa280d3f2"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "6b33480b91a610796dbe1ff76542641a"
server_key_iv: "9c78db8cf574888ef8a705b153610a8c"
server_key_version: "encryptionkey@0"
I20260809 19:28:09.360327 3886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:09.381069 3886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:09.383023 3886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:09.383193 3886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:09.387477 3886 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:33815
I20260809 19:28:09.387516 3957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:33815 every 8 connection(s)
I20260809 19:28:09.387887 3886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:28:09.388945 3958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:09.391249 3958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: Bootstrap starting.
I20260809 19:28:09.391796 3958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:09.392030 3958 log.cc:824] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:09.392719 3958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: No bootstrap required, opened a new log
I20260809 19:28:09.394392 3958 raft_consensus.cc:348] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } }
I20260809 19:28:09.394528 3958 raft_consensus.cc:374] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:09.394552 3958 raft_consensus.cc:729] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 409dc59891924d03b336f94aa280d3f2, State: Initialized, Role: FOLLOWER
I20260809 19:28:09.394711 3958 consensus_queue.cc:260] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } }
I20260809 19:28:09.394790 3958 raft_consensus.cc:388] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:28:09.394814 3958 raft_consensus.cc:482] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:28:09.394861 3958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:09.395643 3958 raft_consensus.cc:504] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } }
I20260809 19:28:09.395767 3958 leader_election.cc:302] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 409dc59891924d03b336f94aa280d3f2; no voters:
I20260809 19:28:09.395964 3958 leader_election.cc:288] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:28:09.395987 3961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:28:09.396106 3961 raft_consensus.cc:686] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 1 LEADER]: Becoming Leader. State: Replica: 409dc59891924d03b336f94aa280d3f2, State: Running, Role: LEADER
I20260809 19:28:09.396205 3961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } }
I20260809 19:28:09.396236 3958 sys_catalog.cc:564] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:28:09.396395 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 3886
I20260809 19:28:09.396486 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:28:09.396744 21831 external_mini_cluster.cc:1428] Setting key 41196221bb8c3a53479435dd4f684e30
I20260809 19:28:09.396626 3963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 409dc59891924d03b336f94aa280d3f2. Latest consensus state: current_term: 1 leader_uuid: "409dc59891924d03b336f94aa280d3f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } } }
I20260809 19:28:09.396618 3962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "409dc59891924d03b336f94aa280d3f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } } }
I20260809 19:28:09.396862 3962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:09.396862 3963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:09.397110 3966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:28:09.397732 3966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:28:09.399463 3966 catalog_manager.cc:1347] Generated new cluster ID: 63e0d12e503649d9bd136cdfc03ab9bf
I20260809 19:28:09.399528 3966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:28:09 dist-test-slave-njxd krb5kdc[3872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:09.409257 3975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:09.397418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35956 (local address 127.21.81.254:33815)
0809 19:28:09.397908 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:09.397920 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:09.397961 (+ 41us) server_negotiation.cc:407] Connection header received
0809 19:28:09.398378 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:09.398385 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:09.398470 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:09.398628 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:09.399416 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.400800 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.401855 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.402537 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.402658 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:09.405540 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:09.405544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:09.405544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:09.405556 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:09.407505 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:09.407969 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:09.407972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:09.407972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:09.408098 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:09.408330 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:09.408330 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:09.408331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:09.408625 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:09.408776 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:09.408888 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 19:28:09.409019 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":90,"threads_started":1}
I20260809 19:28:09.417209 3966 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:28:09.417789 3966 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:28:09.421236 3966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: Generated new TSK 0
I20260809 19:28:09.421432 3966 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:28:09.450642 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:33815
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:09.535386 3985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:09.535576 3985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:09.535600 3985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:09.536749 3985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:09.536791 3985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:09.536844 3985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:28:09.538386 3985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:33815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:09.538662 3985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:09.538899 3985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:09.543974 3990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:09.544023 3991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:09.544011 3985 server_base.cc:1034] running on GCE node
W20260809 19:28:09.544018 3993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:09.544430 3985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:09.544704 3985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:09.545881 3985 hybrid_clock.cc:648] HybridClock initialized: now 1786303689545866 us; error 42 us; skew 500 ppm
Aug 09 19:28:09 dist-test-slave-njxd krb5kdc[3872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303689, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:09.548517 3985 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:28:09.549005 3985 webserver.cc:460] Webserver started at http://127.21.81.193:44703/ using document root <none> and password file <none>
I20260809 19:28:09.549259 3985 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:09.549329 3985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:09.549479 3985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:28:09.550566 3985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "d680939a4eaf4486bb37fc854aa43a7a"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "46d5560b6d092402463220898ed73296"
server_key_iv: "62a54f7c26edd3e8668ebb25de63818d"
server_key_version: "encryptionkey@0"
I20260809 19:28:09.550938 3985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "d680939a4eaf4486bb37fc854aa43a7a"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "46d5560b6d092402463220898ed73296"
server_key_iv: "62a54f7c26edd3e8668ebb25de63818d"
server_key_version: "encryptionkey@0"
I20260809 19:28:09.552275 3985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:28:09.552986 4000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:09.553205 3985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:28:09.553301 3985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "d680939a4eaf4486bb37fc854aa43a7a"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "46d5560b6d092402463220898ed73296"
server_key_iv: "62a54f7c26edd3e8668ebb25de63818d"
server_key_version: "encryptionkey@0"
I20260809 19:28:09.553385 3985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:09.569867 3985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:09.571658 3985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:09.571818 3985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:09.572057 3985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:09.572368 3985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:09.572422 3985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:09.572463 3985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:09.572489 3985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:09.578455 3985 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:36703
I20260809 19:28:09.578508 4131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:36703 every 8 connection(s)
I20260809 19:28:09.578817 3985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:28:09 dist-test-slave-njxd krb5kdc[3872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303689, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:09.586052 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 3985
I20260809 19:28:09.586165 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:28:09.586400 21831 external_mini_cluster.cc:1428] Setting key 6cff7c2147230e286c180aa3a4fd18bc
I20260809 19:28:09.587991 3975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:09.579556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:53069 (local address 127.21.81.254:33815)
0809 19:28:09.579624 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:09.579627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:09.579851 (+ 224us) server_negotiation.cc:407] Connection header received
0809 19:28:09.580307 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:09.580308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:09.580320 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:09.580373 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:09.581261 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.581808 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.582733 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.582931 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.582983 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:09.585090 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:09.585091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:09.585091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:09.585097 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:09.586604 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:09.586985 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:09.586986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:09.586987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:09.587050 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:09.587377 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:09.587377 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:09.587378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:09.587483 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:09.587538 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:09.587854 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 19:28:09.587924 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:28:09.588302 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:09.579534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:53069)
0809 19:28:09.579710 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:09.579747 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:09.580229 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:09.580422 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:09.580425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:09.580989 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:09.581170 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:09.581175 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.581887 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.581887 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:09.582657 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:09.582659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.583005 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.583006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:09.583100 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:09.583645 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:09.583653 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:09.585000 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:09.586680 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:09.586687 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:09.586689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:09.586911 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:09.587144 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:09.587145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:09.587146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:09.587306 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:09.587551 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:09.587558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:09.587718 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 19:28:09.587832 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 19:28:09.588013 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260809 19:28:09.588877 4132 heartbeater.cc:347] Connected to a master server at 127.21.81.254:33815
I20260809 19:28:09.588992 4132 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:09.589185 4132 heartbeater.cc:511] Master 127.21.81.254:33815 requested a full tablet report, sending...
I20260809 19:28:09.589781 3921 ts_manager.cc:194] Registered new tserver with Master: d680939a4eaf4486bb37fc854aa43a7a (127.21.81.193:36703)
I20260809 19:28:09.590389 3921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:53069
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:28:09.612743 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:33815
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:09.694128 4138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:09.694330 4138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:09.694366 4138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:09.695516 4138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:09.695588 4138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:09.695634 4138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:28:09.697202 4138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:33815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:09.697446 4138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:09.697711 4138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:09.702435 4143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:09.702461 4144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:09.702522 4146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:09.702665 4138 server_base.cc:1034] running on GCE node
I20260809 19:28:09.702879 4138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:09.703130 4138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:09.704267 4138 hybrid_clock.cc:648] HybridClock initialized: now 1786303689704261 us; error 31 us; skew 500 ppm
Aug 09 19:28:09 dist-test-slave-njxd krb5kdc[3872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303689, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:09.706799 4138 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:28:09.707233 4138 webserver.cc:460] Webserver started at http://127.21.81.194:41671/ using document root <none> and password file <none>
I20260809 19:28:09.707412 4138 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:09.707450 4138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:09.707535 4138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:28:09.708541 4138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "36410b54d7114a78afdaae0a02d1c95a"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "00d91be06c7b060dc6f0d1f4a2ffee84"
server_key_iv: "9514397819128cb3884446b393152076"
server_key_version: "encryptionkey@0"
I20260809 19:28:09.708845 4138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "36410b54d7114a78afdaae0a02d1c95a"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "00d91be06c7b060dc6f0d1f4a2ffee84"
server_key_iv: "9514397819128cb3884446b393152076"
server_key_version: "encryptionkey@0"
I20260809 19:28:09.710186 4138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:28:09.710879 4155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:09.711058 4138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:28:09.711162 4138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "36410b54d7114a78afdaae0a02d1c95a"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "00d91be06c7b060dc6f0d1f4a2ffee84"
server_key_iv: "9514397819128cb3884446b393152076"
server_key_version: "encryptionkey@0"
I20260809 19:28:09.711416 4138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:09.730736 4138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:09.733317 4138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:09.733489 4138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:09.733784 4138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:09.734169 4138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:09.734252 4138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:09.734308 4138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:09.734339 4138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:09.740932 4138 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:38359
I20260809 19:28:09.740978 4278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:38359 every 8 connection(s)
I20260809 19:28:09.741288 4138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:28:09 dist-test-slave-njxd krb5kdc[3872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303689, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:09.747670 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 4138
I20260809 19:28:09.747773 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:28:09.747995 21831 external_mini_cluster.cc:1428] Setting key 2af331ca46512c27ecdafbde88d5c4ae
I20260809 19:28:09.751170 3975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:09.742046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:34523 (local address 127.21.81.254:33815)
0809 19:28:09.742118 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:09.742120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:09.742325 (+ 205us) server_negotiation.cc:407] Connection header received
0809 19:28:09.742872 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:09.742873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:09.742887 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:09.742953 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:09.743932 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.744447 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.745455 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.745750 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.745829 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:09.747957 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:09.747959 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:09.747959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:09.747968 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:09.749524 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:09.750062 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:09.750063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:09.750063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:09.750119 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:09.750392 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:09.750392 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:09.750393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:09.750538 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:09.750646 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:09.750925 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 19:28:09.751031 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 19:28:09.751194 4281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:09.742013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:34523)
0809 19:28:09.742203 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:09.742241 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:09.742760 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:09.743031 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:09.743034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:09.743615 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:09.743812 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:09.743817 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.744548 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.744549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:09.745354 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:09.745357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.745838 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.745840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:09.745967 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:09.746389 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:09.746397 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:09.747842 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:09.749629 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:09.749637 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:09.749639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:09.749981 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:09.750182 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:09.750183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:09.750183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:09.750327 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:09.750646 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:09.750654 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:09.750793 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 19:28:09.750878 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 19:28:09.751019 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":98,"threads_started":1}
I20260809 19:28:09.751706 4279 heartbeater.cc:347] Connected to a master server at 127.21.81.254:33815
I20260809 19:28:09.751799 4279 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:09.751987 4279 heartbeater.cc:511] Master 127.21.81.254:33815 requested a full tablet report, sending...
I20260809 19:28:09.752358 3921 ts_manager.cc:194] Registered new tserver with Master: 36410b54d7114a78afdaae0a02d1c95a (127.21.81.194:38359)
I20260809 19:28:09.752879 3921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:34523
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:28:09.774642 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:33815
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:09.856285 4284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:09.856600 4284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:09.856645 4284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:09.857839 4284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:09.857910 4284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:09.857957 4284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:28:09.859440 4284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:33815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:09.859714 4284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:09.860015 4284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:09.865082 4290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:09.865108 4292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:09.865111 4289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:09.865360 4284 server_base.cc:1034] running on GCE node
I20260809 19:28:09.865702 4284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:09.865948 4284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:09.867107 4284 hybrid_clock.cc:648] HybridClock initialized: now 1786303689867090 us; error 30 us; skew 500 ppm
Aug 09 19:28:09 dist-test-slave-njxd krb5kdc[3872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303689, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:09.869551 4284 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:28:09.870065 4284 webserver.cc:460] Webserver started at http://127.21.81.195:41123/ using document root <none> and password file <none>
I20260809 19:28:09.870321 4284 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:09.870390 4284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:09.870527 4284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:28:09.871546 4284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "4ae7c217444f433cba51e7867df42b13"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "ebafe32b1d049dada95916fd43afec84"
server_key_iv: "c9e2a1b36fabed894b1290e20d0e0195"
server_key_version: "encryptionkey@0"
I20260809 19:28:09.871899 4284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "4ae7c217444f433cba51e7867df42b13"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "ebafe32b1d049dada95916fd43afec84"
server_key_iv: "c9e2a1b36fabed894b1290e20d0e0195"
server_key_version: "encryptionkey@0"
I20260809 19:28:09.873220 4284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:28:09.874087 4299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:09.874258 4284 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:28:09.874357 4284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "4ae7c217444f433cba51e7867df42b13"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "ebafe32b1d049dada95916fd43afec84"
server_key_iv: "c9e2a1b36fabed894b1290e20d0e0195"
server_key_version: "encryptionkey@0"
I20260809 19:28:09.874444 4284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:09.889510 4284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:09.891494 4284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:09.891660 4284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:09.891901 4284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:09.892220 4284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:09.892277 4284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:09.892318 4284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:09.892347 4284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:09.898751 4284 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:44955
I20260809 19:28:09.898795 4421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:44955 every 8 connection(s)
I20260809 19:28:09.899099 4284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:28:09.899374 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 4284
I20260809 19:28:09.899492 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:28:09.899762 21831 external_mini_cluster.cc:1428] Setting key c185c901372eb78783733cd76985c6ae
Aug 09 19:28:09 dist-test-slave-njxd krb5kdc[3872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303689, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:09.908541 3975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:09.899922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:58133 (local address 127.21.81.254:33815)
0809 19:28:09.899978 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:09.899980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:09.900170 (+ 190us) server_negotiation.cc:407] Connection header received
0809 19:28:09.900700 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:09.900701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:09.900713 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:09.900765 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:09.901680 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.902185 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.903417 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.903640 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.903723 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:09.905958 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:09.905960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:09.905960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:09.905969 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:09.907160 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:09.907566 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:09.907568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:09.907568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:09.907648 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:09.907902 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:09.907903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:09.907903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:09.908020 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:09.908099 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:09.908373 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 19:28:09.908452 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 19:28:09.908721 4424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:09.899888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:58133)
0809 19:28:09.900056 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:09.900098 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:09.900585 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:09.900829 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:09.900833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:09.901386 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:09.901556 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:09.901560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.902293 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.902294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:09.903129 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:09.903131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.903735 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.903736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:09.903874 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:09.904369 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:09.904377 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:09.905858 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:09.907246 (+ 1388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:09.907254 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:09.907256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:09.907490 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:09.907727 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:09.907728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:09.907728 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:09.907838 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:09.908094 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:09.908101 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:09.908238 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 19:28:09.908339 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 19:28:09.908495 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260809 19:28:09.909253 4422 heartbeater.cc:347] Connected to a master server at 127.21.81.254:33815
I20260809 19:28:09.909361 4422 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:09.909562 4422 heartbeater.cc:511] Master 127.21.81.254:33815 requested a full tablet report, sending...
I20260809 19:28:09.909988 3921 ts_manager.cc:194] Registered new tserver with Master: 4ae7c217444f433cba51e7867df42b13 (127.21.81.195:44955)
I20260809 19:28:09.910370 3921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:58133
I20260809 19:28:09.911270 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:28:09.918268 3975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:09.912558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35960 (local address 127.21.81.254:33815)
0809 19:28:09.912604 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:09.912606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:09.912667 (+ 61us) server_negotiation.cc:407] Connection header received
0809 19:28:09.912742 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:09.912743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:09.912752 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:09.912795 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:09.913335 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.913937 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.914722 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.914915 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.914992 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:09.916059 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:09.916060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:09.916061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:09.916067 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:09.917105 (+ 1038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:09.917448 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:09.917449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:09.917449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:09.917508 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:09.917731 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:09.917732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:09.917733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:09.917877 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:09.917996 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:09.918096 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 19:28:09.918173 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:28:09.919641 3921 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:28:09.921339 3921 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:28:09.927944 4438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:09.923677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:38359 (local address 127.0.0.1:57960)
0809 19:28:09.924067 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:09.924108 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:09.924189 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:09.924541 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:09.924544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:09.924548 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:09.924614 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:09.924616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.925521 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.925522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:09.926721 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:09.926724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.927629 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.927630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:09.927722 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:09.927736 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 19:28:09.927808 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 19:28:09.927861 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":3,"thread_start_us":47,"threads_started":1}
I20260809 19:28:09.928154 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:09.923941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46748 (local address 127.21.81.195:44955)
0809 19:28:09.924099 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:09.924103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:09.924402 (+ 299us) server_negotiation.cc:407] Connection header received
0809 19:28:09.924453 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:09.924455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:09.924469 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:09.924536 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:09.924742 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.925878 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.927129 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.927636 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.927705 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:09.927761 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:09.927985 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 19:28:09.928034 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260809 19:28:09.928187 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:09.923862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:44955 (local address 127.0.0.1:46748)
0809 19:28:09.924295 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:09.924302 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:09.924342 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:09.924597 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:09.924597 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:09.924599 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:09.924656 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:09.924676 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.926001 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.926001 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:09.927047 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:09.927049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.927797 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.927798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:09.927851 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:09.927854 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:28:09.927900 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 19:28:09.927931 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":38,"thread_start_us":71,"threads_started":1}
I20260809 19:28:09.928304 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:09.923676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:36703 (local address 127.0.0.1:60294)
0809 19:28:09.924067 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:09.924108 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:09.924189 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:09.924541 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:09.924544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:09.924546 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:09.924613 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:09.924615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.926004 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.926005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:09.927068 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:09.927071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.928041 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.928042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:09.928103 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:09.928107 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:28:09.928169 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:28:09.928217 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":52,"thread_start_us":85,"threads_started":1}
I20260809 19:28:09.927984 4437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:09.923808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57960 (local address 127.21.81.194:38359)
0809 19:28:09.924005 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:09.924009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:09.924201 (+ 192us) server_negotiation.cc:407] Connection header received
0809 19:28:09.924356 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:09.924358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:09.924376 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:09.924457 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:09.924714 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.925417 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.926811 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.927537 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.927612 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:09.927677 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:09.927835 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 19:28:09.927882 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
I20260809 19:28:09.928615 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:09.923808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60294 (local address 127.21.81.193:36703)
0809 19:28:09.924005 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:09.924009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:09.924273 (+ 264us) server_negotiation.cc:407] Connection header received
0809 19:28:09.924325 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:09.924328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:09.924346 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:09.924425 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:09.924701 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.925895 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.927175 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:09.927889 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:09.927954 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:09.928004 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:09.928489 (+ 485us) server_negotiation.cc:299] Negotiation successful
0809 19:28:09.928534 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":89,"threads_started":1}
I20260809 19:28:09.929096 4206 tablet_service.cc:1467] Processing CreateTablet for tablet 3fd140fc3c184b229bb31439df33053a (DEFAULT_TABLE table=test-table [id=9e6e8d79249248b2ae5a08d2474b2b31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:28:09.929100 4352 tablet_service.cc:1467] Processing CreateTablet for tablet 3fd140fc3c184b229bb31439df33053a (DEFAULT_TABLE table=test-table [id=9e6e8d79249248b2ae5a08d2474b2b31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:28:09.929312 4051 tablet_service.cc:1467] Processing CreateTablet for tablet 3fd140fc3c184b229bb31439df33053a (DEFAULT_TABLE table=test-table [id=9e6e8d79249248b2ae5a08d2474b2b31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:28:09.929450 4352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fd140fc3c184b229bb31439df33053a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:09.929435 4206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fd140fc3c184b229bb31439df33053a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:09.929639 4051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fd140fc3c184b229bb31439df33053a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:09.931973 4440 tablet_bootstrap.cc:492] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a: Bootstrap starting.
I20260809 19:28:09.932628 4440 tablet_bootstrap.cc:654] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:09.932767 4441 tablet_bootstrap.cc:492] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13: Bootstrap starting.
I20260809 19:28:09.933483 4441 tablet_bootstrap.cc:654] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:09.933558 4440 log.cc:824] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:09.934125 4441 log.cc:824] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:09.934374 4440 tablet_bootstrap.cc:492] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a: No bootstrap required, opened a new log
I20260809 19:28:09.934444 4440 ts_tablet_manager.cc:1397] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 19:28:09.934885 4441 tablet_bootstrap.cc:492] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13: No bootstrap required, opened a new log
I20260809 19:28:09.934959 4441 ts_tablet_manager.cc:1397] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:28:09.936484 4440 raft_consensus.cc:348] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } }
I20260809 19:28:09.936662 4440 raft_consensus.cc:374] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:09.936697 4440 raft_consensus.cc:729] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36410b54d7114a78afdaae0a02d1c95a, State: Initialized, Role: FOLLOWER
I20260809 19:28:09.936820 4440 consensus_queue.cc:260] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } }
I20260809 19:28:09.936856 4441 raft_consensus.cc:348] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } }
I20260809 19:28:09.937012 4441 raft_consensus.cc:374] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:09.937098 4441 raft_consensus.cc:729] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ae7c217444f433cba51e7867df42b13, State: Initialized, Role: FOLLOWER
I20260809 19:28:09.937121 4440 ts_tablet_manager.cc:1428] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:28:09.937144 4279 heartbeater.cc:503] Master 127.21.81.254:33815 was elected leader, sending a full tablet report...
I20260809 19:28:09.934000 4442 tablet_bootstrap.cc:492] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: Bootstrap starting.
I20260809 19:28:09.937304 4441 consensus_queue.cc:260] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } }
I20260809 19:28:09.937649 4441 ts_tablet_manager.cc:1428] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:28:09.937739 4422 heartbeater.cc:503] Master 127.21.81.254:33815 was elected leader, sending a full tablet report...
I20260809 19:28:09.938189 4442 tablet_bootstrap.cc:654] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:09.938799 4442 log.cc:824] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:09.939579 4442 tablet_bootstrap.cc:492] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: No bootstrap required, opened a new log
I20260809 19:28:09.939631 4442 ts_tablet_manager.cc:1397] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260809 19:28:09.941040 4442 raft_consensus.cc:348] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } }
I20260809 19:28:09.941155 4442 raft_consensus.cc:374] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:09.941176 4442 raft_consensus.cc:729] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d680939a4eaf4486bb37fc854aa43a7a, State: Initialized, Role: FOLLOWER
I20260809 19:28:09.941282 4442 consensus_queue.cc:260] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } }
I20260809 19:28:09.941511 4442 ts_tablet_manager.cc:1428] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:28:09.941591 4132 heartbeater.cc:503] Master 127.21.81.254:33815 was elected leader, sending a full tablet report...
W20260809 19:28:09.991864 4280 tablet.cc:2369] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:28:10.013967 4449 raft_consensus.cc:482] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:28:10.014204 4449 raft_consensus.cc:504] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } }
I20260809 19:28:10.014717 4449 leader_election.cc:288] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36410b54d7114a78afdaae0a02d1c95a (127.21.81.194:38359), 4ae7c217444f433cba51e7867df42b13 (127.21.81.195:44955)
I20260809 19:28:10.017877 4437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.014981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:56639 (local address 127.21.81.194:38359)
0809 19:28:10.015035 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:10.015037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:10.015051 (+ 14us) server_negotiation.cc:407] Connection header received
0809 19:28:10.015088 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:10.015089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:10.015103 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:10.015162 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:10.015399 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.015867 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.016979 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.017501 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.017556 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.017564 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:10.017791 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 19:28:10.017825 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:28:10.017899 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.014886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:38359 (local address 127.21.81.193:56639)
0809 19:28:10.014951 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:10.014964 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:10.015024 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:10.015180 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:10.015181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:10.015183 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:10.015302 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.015304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.015945 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.015946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.016854 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.016856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.017615 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.017616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.017709 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.017713 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:28:10.017778 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 19:28:10.017823 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 19:28:10.018281 4226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd140fc3c184b229bb31439df33053a" candidate_uuid: "d680939a4eaf4486bb37fc854aa43a7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36410b54d7114a78afdaae0a02d1c95a" is_pre_election: true
I20260809 19:28:10.018324 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.015009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:44955 (local address 127.21.81.193:38911)
0809 19:28:10.015248 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:10.015257 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:10.015329 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:10.015483 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:10.015484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:10.015486 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:10.015571 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.015572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.016349 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.016350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.017233 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.017235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.018027 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.018028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.018099 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.018102 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:28:10.018208 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 19:28:10.018251 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":150,"threads_started":1}
I20260809 19:28:10.018334 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.015097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:38911 (local address 127.21.81.195:44955)
0809 19:28:10.015140 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:10.015142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:10.015331 (+ 189us) server_negotiation.cc:407] Connection header received
0809 19:28:10.015390 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:10.015392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:10.015402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:10.015477 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:10.015646 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.016255 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.017346 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.017910 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.017981 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.017992 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:10.018221 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 19:28:10.018270 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:28:10.018460 4226 raft_consensus.cc:2454] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d680939a4eaf4486bb37fc854aa43a7a in term 0.
I20260809 19:28:10.018690 4004 leader_election.cc:302] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36410b54d7114a78afdaae0a02d1c95a, d680939a4eaf4486bb37fc854aa43a7a; no voters:
I20260809 19:28:10.018725 4372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd140fc3c184b229bb31439df33053a" candidate_uuid: "d680939a4eaf4486bb37fc854aa43a7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae7c217444f433cba51e7867df42b13" is_pre_election: true
I20260809 19:28:10.018895 4372 raft_consensus.cc:2454] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d680939a4eaf4486bb37fc854aa43a7a in term 0.
I20260809 19:28:10.018886 4449 raft_consensus.cc:2781] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:28:10.018944 4449 raft_consensus.cc:482] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:28:10.018973 4449 raft_consensus.cc:3037] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:10.019739 4449 raft_consensus.cc:504] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } }
I20260809 19:28:10.019898 4449 leader_election.cc:288] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [CANDIDATE]: Term 1 election: Requested vote from peers 36410b54d7114a78afdaae0a02d1c95a (127.21.81.194:38359), 4ae7c217444f433cba51e7867df42b13 (127.21.81.195:44955)
I20260809 19:28:10.020071 4372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd140fc3c184b229bb31439df33053a" candidate_uuid: "d680939a4eaf4486bb37fc854aa43a7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae7c217444f433cba51e7867df42b13"
I20260809 19:28:10.020073 4226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd140fc3c184b229bb31439df33053a" candidate_uuid: "d680939a4eaf4486bb37fc854aa43a7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36410b54d7114a78afdaae0a02d1c95a"
I20260809 19:28:10.020153 4372 raft_consensus.cc:3037] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:10.020153 4226 raft_consensus.cc:3037] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:10.021111 4226 raft_consensus.cc:2454] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d680939a4eaf4486bb37fc854aa43a7a in term 1.
I20260809 19:28:10.021106 4372 raft_consensus.cc:2454] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d680939a4eaf4486bb37fc854aa43a7a in term 1.
I20260809 19:28:10.021260 4004 leader_election.cc:302] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36410b54d7114a78afdaae0a02d1c95a, d680939a4eaf4486bb37fc854aa43a7a; no voters:
I20260809 19:28:10.021351 4449 raft_consensus.cc:2781] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:28:10.021502 4449 raft_consensus.cc:686] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [term 1 LEADER]: Becoming Leader. State: Replica: d680939a4eaf4486bb37fc854aa43a7a, State: Running, Role: LEADER
I20260809 19:28:10.021626 4449 consensus_queue.cc:237] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } }
I20260809 19:28:10.022516 3920 catalog_manager.cc:5495] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a reported cstate change: term changed from 0 to 1, leader changed from <none> to d680939a4eaf4486bb37fc854aa43a7a (127.21.81.193). New cstate: current_term: 1 leader_uuid: "d680939a4eaf4486bb37fc854aa43a7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } health_report { overall_health: UNKNOWN } } }
I20260809 19:28:10.046172 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.044016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60298 (local address 127.21.81.193:36703)
0809 19:28:10.044076 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:10.044078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:10.044101 (+ 23us) server_negotiation.cc:407] Connection header received
0809 19:28:10.044139 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:10.044139 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:10.044155 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:10.044212 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:28:10.044330 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.044786 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.045513 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.045721 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.045767 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.045898 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:28:10.045984 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:28:10.046041 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:10.046090 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 19:28:10.046106 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:28:10.048027 4372 raft_consensus.cc:1261] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13 [term 1 FOLLOWER]: Refusing update from remote peer d680939a4eaf4486bb37fc854aa43a7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:28:10.048027 4226 raft_consensus.cc:1261] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a [term 1 FOLLOWER]: Refusing update from remote peer d680939a4eaf4486bb37fc854aa43a7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:28:10.048381 4452 consensus_queue.cc:1035] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:28:10.048511 4449 consensus_queue.cc:1035] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:28:10.053905 3918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:28:10.053995 3918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:28:10.054759 3918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: Sending DeleteTablet for 3 replicas of tablet 3fd140fc3c184b229bb31439df33053a
I20260809 19:28:10.055091 4051 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fd140fc3c184b229bb31439df33053a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:10 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:60294
I20260809 19:28:10.055151 4206 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fd140fc3c184b229bb31439df33053a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:10 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:57960
I20260809 19:28:10.055151 4352 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fd140fc3c184b229bb31439df33053a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:10 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:46748
I20260809 19:28:10.055395 4466 tablet_replica.cc:330] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: stopping tablet replica
I20260809 19:28:10.055444 4467 tablet_replica.cc:330] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a: stopping tablet replica
I20260809 19:28:10.055454 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 3985
I20260809 19:28:10.060923 4468 tablet_replica.cc:330] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13: stopping tablet replica
I20260809 19:28:10.061071 4468 raft_consensus.cc:2229] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:28:10.061197 4468 raft_consensus.cc:2258] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:28:10.061321 4467 raft_consensus.cc:2229] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:28:10.061463 4467 raft_consensus.cc:2258] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:28:10.061992 4467 ts_tablet_manager.cc:1905] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:28:10.062644 4468 ts_tablet_manager.cc:1905] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:28:10.064026 4467 ts_tablet_manager.cc:1918] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:28:10.064102 4467 log.cc:1196] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/3fd140fc3c184b229bb31439df33053a
I20260809 19:28:10.064409 4467 ts_tablet_manager.cc:1939] T 3fd140fc3c184b229bb31439df33053a P 36410b54d7114a78afdaae0a02d1c95a: Deleting consensus metadata
I20260809 19:28:10.064739 4468 ts_tablet_manager.cc:1918] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:28:10.064844 4468 log.cc:1196] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/3fd140fc3c184b229bb31439df33053a
I20260809 19:28:10.064831 3906 catalog_manager.cc:4841] TS 36410b54d7114a78afdaae0a02d1c95a (127.21.81.194:38359): tablet 3fd140fc3c184b229bb31439df33053a (table test-table [id=9e6e8d79249248b2ae5a08d2474b2b31]) successfully deleted
I20260809 19:28:10.065150 4468 ts_tablet_manager.cc:1939] T 3fd140fc3c184b229bb31439df33053a P 4ae7c217444f433cba51e7867df42b13: Deleting consensus metadata
I20260809 19:28:10.065562 3906 catalog_manager.cc:4841] TS 4ae7c217444f433cba51e7867df42b13 (127.21.81.195:44955): tablet 3fd140fc3c184b229bb31439df33053a (table test-table [id=9e6e8d79249248b2ae5a08d2474b2b31]) successfully deleted
W20260809 19:28:10.067520 3904 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:36703 (error 108)
I20260809 19:28:10.067734 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 4138
I20260809 19:28:10.068763 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.068613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:36703 (local address 127.0.0.1:60308)
0809 19:28:10.068676 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:10.068721 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:36703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 19:28:10.068889 3904 catalog_manager.cc:4584] TS d680939a4eaf4486bb37fc854aa43a7a (127.21.81.193:36703): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3fd140fc3c184b229bb31439df33053a: Network error: Client connection negotiation failed: client connection to 127.21.81.193:36703: connect: Connection refused (error 111)
I20260809 19:28:10.074012 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 4284
I20260809 19:28:10.079743 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 3886
I20260809 19:28:10.086568 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:33815
--webserver_interface=127.21.81.254
--webserver_port=33175
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:33815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:10.168577 4470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:10.168772 4470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:10.168795 4470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:10.169981 4470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:28:10.170022 4470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:10.170037 4470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:10.170097 4470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:28:10.170123 4470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:28:10.171552 4470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:33815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:33815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=33175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:10.171808 4470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:10.172037 4470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:10.181154 4476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:10.181222 4477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:10.181360 4470 server_base.cc:1034] running on GCE node
W20260809 19:28:10.181227 4480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:10.182070 4470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:10.182382 4470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:10.183553 4470 hybrid_clock.cc:648] HybridClock initialized: now 1786303690183541 us; error 45 us; skew 500 ppm
Aug 09 19:28:10 dist-test-slave-njxd krb5kdc[3872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:10.186081 4470 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:28:10.186585 4470 webserver.cc:460] Webserver started at http://127.21.81.254:33175/ using document root <none> and password file <none>
I20260809 19:28:10.186849 4470 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:10.186930 4470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:10.188293 4470 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:10.188963 4488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:10.189128 4470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:28:10.189241 4470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "409dc59891924d03b336f94aa280d3f2"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "6b33480b91a610796dbe1ff76542641a"
server_key_iv: "9c78db8cf574888ef8a705b153610a8c"
server_key_version: "encryptionkey@0"
I20260809 19:28:10.189683 4470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:10.198132 4470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:10.199882 4470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:10.200039 4470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:10.204228 4470 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:33815
I20260809 19:28:10.204259 4541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:33815 every 8 connection(s)
I20260809 19:28:10.204622 4470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:28:10.206236 4542 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:28:10.206859 4542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: Bootstrap starting.
I20260809 19:28:10.211305 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 4470
I20260809 19:28:10.211577 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:36703
--local_ip_for_outbound_sockets=127.21.81.193
--tserver_master_addrs=127.21.81.254:33815
--webserver_port=44703
--webserver_interface=127.21.81.193
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:28:10.216948 4542 log.cc:824] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:10.218986 4542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:28:10.219311 4542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: Bootstrap complete.
I20260809 19:28:10.221913 4542 raft_consensus.cc:348] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } }
I20260809 19:28:10.222136 4542 raft_consensus.cc:729] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 409dc59891924d03b336f94aa280d3f2, State: Initialized, Role: FOLLOWER
I20260809 19:28:10.222309 4542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } }
I20260809 19:28:10.222380 4542 raft_consensus.cc:388] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:28:10.222450 4542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:28:10.222504 4542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:28:10.223518 4542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } }
I20260809 19:28:10.223630 4542 leader_election.cc:302] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 409dc59891924d03b336f94aa280d3f2; no voters:
I20260809 19:28:10.223800 4542 leader_election.cc:288] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 19:28:10.223848 4547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:28:10.224009 4547 raft_consensus.cc:686] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 2 LEADER]: Becoming Leader. State: Replica: 409dc59891924d03b336f94aa280d3f2, State: Running, Role: LEADER
I20260809 19:28:10.224080 4542 sys_catalog.cc:564] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:28:10.224155 4547 consensus_queue.cc:237] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } }
I20260809 19:28:10.224442 4548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "409dc59891924d03b336f94aa280d3f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } } }
I20260809 19:28:10.224614 4548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:10.224438 4547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 409dc59891924d03b336f94aa280d3f2. Latest consensus state: current_term: 2 leader_uuid: "409dc59891924d03b336f94aa280d3f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } } }
I20260809 19:28:10.224980 4557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:28:10.225051 4547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:10.225896 4557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:28:10.226083 4557 catalog_manager.cc:1259] Loaded cluster ID: 63e0d12e503649d9bd136cdfc03ab9bf
I20260809 19:28:10.226266 4557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:28:10.226543 4557 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 19:28:10.231587 4557 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 19:28:10.234136 4557 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:28:10.234359 4557 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: Loaded TSK: 0
I20260809 19:28:10.234586 4557 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 19:28:10.300048 4545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:10.300257 4545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:10.300282 4545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:10.301473 4545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:10.301517 4545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:10.301543 4545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:28:10.303023 4545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:36703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=44703
--tserver_master_addrs=127.21.81.254:33815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:10.303267 4545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:10.303524 4545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:10.309742 4573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:10.309809 4572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:10.309823 4545 server_base.cc:1034] running on GCE node
W20260809 19:28:10.309962 4575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:10.310180 4545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:10.310451 4545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:10.311619 4545 hybrid_clock.cc:648] HybridClock initialized: now 1786303690311593 us; error 45 us; skew 500 ppm
Aug 09 19:28:10 dist-test-slave-njxd krb5kdc[3872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:10.314460 4545 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:28:10.315143 4545 webserver.cc:460] Webserver started at http://127.21.81.193:44703/ using document root <none> and password file <none>
I20260809 19:28:10.315410 4545 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:10.315492 4545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:10.317320 4545 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 19:28:10.318182 4583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:10.318390 4545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:28:10.318491 4545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "d680939a4eaf4486bb37fc854aa43a7a"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "46d5560b6d092402463220898ed73296"
server_key_iv: "62a54f7c26edd3e8668ebb25de63818d"
server_key_version: "encryptionkey@0"
I20260809 19:28:10.318998 4545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:10.351581 4545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:10.354130 4545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:10.354281 4545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:10.354550 4545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:10.355151 4590 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:28:10.356647 4545 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:28:10.356719 4545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:28:10.356755 4545 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:28:10.357651 4545 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:28:10.357704 4545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:10.357786 4590 tablet_bootstrap.cc:492] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: Bootstrap starting.
I20260809 19:28:10.364475 4545 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:36703
I20260809 19:28:10.364646 4702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:36703 every 8 connection(s)
I20260809 19:28:10.365018 4545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:28:10.368427 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 4545
I20260809 19:28:10.368757 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:38359
--local_ip_for_outbound_sockets=127.21.81.194
--tserver_master_addrs=127.21.81.254:33815
--webserver_port=41671
--webserver_interface=127.21.81.194
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:28:10.369123 4590 log.cc:824] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:10.372130 4590 tablet_bootstrap.cc:492] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:28:10.372485 4590 tablet_bootstrap.cc:492] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: Bootstrap complete.
Aug 09 19:28:10 dist-test-slave-njxd krb5kdc[3872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:10.373827 4590 ts_tablet_manager.cc:1397] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.015s
I20260809 19:28:10.375623 4590 raft_consensus.cc:348] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } }
I20260809 19:28:10.376067 4590 raft_consensus.cc:729] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d680939a4eaf4486bb37fc854aa43a7a, State: Initialized, Role: FOLLOWER
I20260809 19:28:10.376271 4590 consensus_queue.cc:260] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } }
I20260809 19:28:10.376600 4590 ts_tablet_manager.cc:1428] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:28:10.377981 4707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.365888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:56527 (local address 127.21.81.254:33815)
0809 19:28:10.367996 (+ 2108us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:10.368004 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:10.368034 (+ 30us) server_negotiation.cc:407] Connection header received
0809 19:28:10.368145 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:10.368150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:10.368192 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:10.368359 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:10.369211 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.370324 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.371368 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.371763 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.371896 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.374293 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:10.374297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:10.374297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:10.374309 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:10.375973 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:10.376626 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:10.376629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:10.376629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:10.376749 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:10.377008 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:10.377010 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:10.377010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:10.377237 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:10.377340 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:10.377664 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 19:28:10.377789 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2067,"thread_start_us":88,"threads_started":1}
I20260809 19:28:10.377981 4706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.365814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:56527)
0809 19:28:10.366145 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:10.366193 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:10.366992 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:10.368404 (+ 1412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:10.368408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:10.368775 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:10.369081 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.369088 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.370454 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.370456 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.371266 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.371268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.371887 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.371888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.372031 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.372651 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:10.372659 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:10.374184 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:10.376210 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:10.376218 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:10.376220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:10.376519 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:10.376829 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:10.376830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:10.376830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:10.376939 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:10.377332 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:10.377339 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:10.377474 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 19:28:10.377575 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 19:28:10.377789 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":102,"threads_started":1}
I20260809 19:28:10.378779 4703 heartbeater.cc:347] Connected to a master server at 127.21.81.254:33815
I20260809 19:28:10.378870 4703 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:10.379033 4703 heartbeater.cc:511] Master 127.21.81.254:33815 requested a full tablet report, sending...
I20260809 19:28:10.379743 4505 ts_manager.cc:194] Registered new tserver with Master: d680939a4eaf4486bb37fc854aa43a7a (127.21.81.193:36703)
I20260809 19:28:10.380792 4505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:56527
I20260809 19:28:10.382099 4703 heartbeater.cc:503] Master 127.21.81.254:33815 was elected leader, sending a full tablet report...
Aug 09 19:28:10 dist-test-slave-njxd krb5kdc[3872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for kudu/127.21.81.193@KRBTEST.COM
I20260809 19:28:10.387931 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.380336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:36703 (local address 127.0.0.1:60316)
0809 19:28:10.380513 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:10.380548 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:10.380610 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:10.380812 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:10.380818 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:10.381129 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:10.381184 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.381186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.382081 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.382082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.382852 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.382854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.383309 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.383310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.383388 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.383836 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:10.383840 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:10.385110 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:10.386752 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:10.386759 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:10.386761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:10.386982 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:10.387207 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:10.387208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:10.387208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:10.387267 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:10.387550 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:10.387557 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:10.387607 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 19:28:10.387711 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 19:28:10.387834 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260809 19:28:10.387931 4713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.380408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60316 (local address 127.21.81.193:36703)
0809 19:28:10.380589 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:10.380594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:10.380607 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:28:10.380697 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:10.380700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:10.380720 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:10.380803 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:10.381271 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.381999 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.382946 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.383230 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.383308 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.385202 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:10.385209 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:10.385210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:10.385217 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:10.386635 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:10.387067 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:10.387070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:10.387070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:10.387130 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:10.387343 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:10.387344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:10.387344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:10.387480 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:10.387570 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:10.387752 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 19:28:10.387839 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":67,"threads_started":1}
I20260809 19:28:10.388525 4635 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fd140fc3c184b229bb31439df33053a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:10 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:60316
I20260809 19:28:10.389101 4714 tablet_replica.cc:330] stopping tablet replica
I20260809 19:28:10.389225 4714 raft_consensus.cc:2229] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:28:10.389320 4714 raft_consensus.cc:2258] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:28:10.389657 4714 ts_tablet_manager.cc:1905] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:28:10.391733 4714 ts_tablet_manager.cc:1918] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:28:10.391819 4714 log.cc:1196] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal/wals/3fd140fc3c184b229bb31439df33053a
I20260809 19:28:10.392046 4714 ts_tablet_manager.cc:1939] T 3fd140fc3c184b229bb31439df33053a P d680939a4eaf4486bb37fc854aa43a7a: Deleting consensus metadata
I20260809 19:28:10.392534 4490 catalog_manager.cc:4841] TS d680939a4eaf4486bb37fc854aa43a7a (127.21.81.193:36703): tablet 3fd140fc3c184b229bb31439df33053a (table test-table [id=9e6e8d79249248b2ae5a08d2474b2b31]) successfully deleted
W20260809 19:28:10.457008 4709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:10.457221 4709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:10.457245 4709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:10.458365 4709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:10.458402 4709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:10.458428 4709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:28:10.459811 4709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:38359
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=41671
--tserver_master_addrs=127.21.81.254:33815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:10.459987 4709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:10.460165 4709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:10.465180 4720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:10.465157 4721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:10.466341 4709 server_base.cc:1034] running on GCE node
W20260809 19:28:10.465179 4723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:10.466670 4709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:10.466921 4709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:10.468093 4709 hybrid_clock.cc:648] HybridClock initialized: now 1786303690468087 us; error 36 us; skew 500 ppm
Aug 09 19:28:10 dist-test-slave-njxd krb5kdc[3872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:10.470688 4709 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:28:10.471140 4709 webserver.cc:460] Webserver started at http://127.21.81.194:41671/ using document root <none> and password file <none>
I20260809 19:28:10.471366 4709 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:10.471448 4709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:10.472761 4709 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:10.473372 4730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:10.473546 4709 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:28:10.473680 4709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "36410b54d7114a78afdaae0a02d1c95a"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "00d91be06c7b060dc6f0d1f4a2ffee84"
server_key_iv: "9514397819128cb3884446b393152076"
server_key_version: "encryptionkey@0"
I20260809 19:28:10.474033 4709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:10.484660 4709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:10.486950 4709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:10.487110 4709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:10.487339 4709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:10.487645 4709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:10.487701 4709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:10.487742 4709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:10.487771 4709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:10.493875 4709 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:38359
I20260809 19:28:10.493920 4855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:38359 every 8 connection(s)
I20260809 19:28:10.494253 4709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:28:10.495992 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 4709
I20260809 19:28:10.496265 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:44955
--local_ip_for_outbound_sockets=127.21.81.195
--tserver_master_addrs=127.21.81.254:33815
--webserver_port=41123
--webserver_interface=127.21.81.195
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:28:10 dist-test-slave-njxd krb5kdc[3872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:10.504781 4707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.494891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:52689 (local address 127.21.81.254:33815)
0809 19:28:10.494954 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:10.494957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:10.495230 (+ 273us) server_negotiation.cc:407] Connection header received
0809 19:28:10.495775 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:10.495776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:10.495789 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:10.495872 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:10.496965 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.497743 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.498741 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.499085 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.499165 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.501770 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:10.501772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:10.501772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:10.501779 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:10.503224 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:10.503761 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:10.503761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:10.503762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:10.503815 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:10.504113 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:10.504114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:10.504114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:10.504219 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:10.504268 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:10.504634 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 19:28:10.504714 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 19:28:10.505092 4858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.494894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:52689)
0809 19:28:10.495075 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:10.495131 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:10.495686 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:10.495942 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:10.495946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:10.496687 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:10.496870 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.496875 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.497831 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.497832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.498649 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.498651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.499171 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.499172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.499272 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.499809 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:10.499821 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:10.501670 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:10.503317 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:10.503325 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:10.503327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:10.503678 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:10.503884 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:10.503885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:10.503886 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:10.504040 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:10.504286 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:10.504294 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:10.504481 (+ 187us) client_negotiation.cc:769] Sending connection context
0809 19:28:10.504612 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 19:28:10.504806 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
I20260809 19:28:10.505651 4856 heartbeater.cc:347] Connected to a master server at 127.21.81.254:33815
I20260809 19:28:10.505766 4856 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:10.505987 4856 heartbeater.cc:511] Master 127.21.81.254:33815 requested a full tablet report, sending...
I20260809 19:28:10.506430 4505 ts_manager.cc:194] Registered new tserver with Master: 36410b54d7114a78afdaae0a02d1c95a (127.21.81.194:38359)
I20260809 19:28:10.506808 4505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:52689
W20260809 19:28:10.580834 4859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:10.581032 4859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:10.581054 4859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:10.582288 4859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:10.582329 4859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:10.582378 4859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:28:10.583843 4859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:44955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=41123
--tserver_master_addrs=127.21.81.254:33815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:10.584108 4859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:10.584364 4859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:10.589313 4866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:10.589323 4867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:10.589604 4859 server_base.cc:1034] running on GCE node
W20260809 19:28:10.589392 4869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:10.589898 4859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:10.590153 4859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:10.591325 4859 hybrid_clock.cc:648] HybridClock initialized: now 1786303690591320 us; error 34 us; skew 500 ppm
Aug 09 19:28:10 dist-test-slave-njxd krb5kdc[3872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:10.594123 4859 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:28:10.594607 4859 webserver.cc:460] Webserver started at http://127.21.81.195:41123/ using document root <none> and password file <none>
I20260809 19:28:10.594854 4859 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:10.594934 4859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:10.596288 4859 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:10.596894 4876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:10.597064 4859 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:10.597162 4859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "4ae7c217444f433cba51e7867df42b13"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "ebafe32b1d049dada95916fd43afec84"
server_key_iv: "c9e2a1b36fabed894b1290e20d0e0195"
server_key_version: "encryptionkey@0"
I20260809 19:28:10.597512 4859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:10.606174 4859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:10.608008 4859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:10.608170 4859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:10.608402 4859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:10.608706 4859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:10.608760 4859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:10.608803 4859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:10.608830 4859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:10.615139 4859 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:44955
I20260809 19:28:10.615187 4998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:44955 every 8 connection(s)
I20260809 19:28:10.615543 4859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:28:10 dist-test-slave-njxd krb5kdc[3872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:10.622319 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 4859
I20260809 19:28:10.625501 4707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.616278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:54847 (local address 127.21.81.254:33815)
0809 19:28:10.616359 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:10.616361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:10.616611 (+ 250us) server_negotiation.cc:407] Connection header received
0809 19:28:10.617142 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:10.617143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:10.617154 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:10.617213 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:10.618234 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.618742 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.619697 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.619888 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.619942 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.622110 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:10.622111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:10.622111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:10.622118 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:10.624018 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:10.624514 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:10.624515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:10.624515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:10.624567 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:10.624842 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:10.624843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:10.624843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:10.624968 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:10.625043 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:10.625321 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 19:28:10.625404 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 19:28:10.625692 5002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.616282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:54847)
0809 19:28:10.616446 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:10.616488 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:10.617044 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:10.617273 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:10.617276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:10.617930 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:10.618123 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.618128 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.618829 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.618829 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.619610 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.619613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.619978 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.619979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.620082 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.620581 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:10.620589 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:10.622013 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:10.624073 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:10.624081 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:10.624083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:10.624442 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:10.624629 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:10.624630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:10.624630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:10.624779 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:10.625035 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:10.625073 (+ 38us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:10.625206 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 19:28:10.625290 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 19:28:10.625451 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260809 19:28:10.626222 4999 heartbeater.cc:347] Connected to a master server at 127.21.81.254:33815
I20260809 19:28:10.626343 4999 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:10.626530 4999 heartbeater.cc:511] Master 127.21.81.254:33815 requested a full tablet report, sending...
I20260809 19:28:10.626948 4505 ts_manager.cc:194] Registered new tserver with Master: 4ae7c217444f433cba51e7867df42b13 (127.21.81.195:44955)
I20260809 19:28:10.627311 4505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:54847
I20260809 19:28:10.630148 5004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.622736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35976 (local address 127.21.81.254:33815)
0809 19:28:10.622862 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:10.622864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:10.622898 (+ 34us) server_negotiation.cc:407] Connection header received
0809 19:28:10.622981 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:10.622982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:10.622991 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:10.623046 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:10.623807 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.624677 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.625782 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.626083 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.626146 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.627445 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:10.627446 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:10.627446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:10.627454 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:10.628931 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:10.629310 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:10.629311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:10.629312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:10.629374 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:10.629609 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:10.629610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:10.629610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:10.629725 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:10.629910 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:10.629949 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 19:28:10.630027 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260809 19:28:10.630851 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:28:10.633283 5004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.631203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35986 (local address 127.21.81.254:33815)
0809 19:28:10.631254 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:10.631256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:10.631336 (+ 80us) server_negotiation.cc:407] Connection header received
0809 19:28:10.631387 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:10.631387 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:10.631415 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:10.631469 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:28:10.631551 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.631978 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.632640 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.632817 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.632867 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.632997 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:28:10.633079 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:28:10.633138 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:10.633169 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 19:28:10.633188 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:28:10.633524 4505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:35986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:28:10.635152 4505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:28:10.637063 4635 tablet_service.cc:1467] Processing CreateTablet for tablet 192a7c48148941adbc91db9bb58d32cf (DEFAULT_TABLE table=test-table [id=b8c5711604624af6a33674f9a5e7757f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:28:10.637286 4635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 192a7c48148941adbc91db9bb58d32cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:10.639026 4590 tablet_bootstrap.cc:492] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a: Bootstrap starting.
I20260809 19:28:10.639381 4590 tablet_bootstrap.cc:654] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:10.639982 4590 tablet_bootstrap.cc:492] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a: No bootstrap required, opened a new log
I20260809 19:28:10.640033 4590 ts_tablet_manager.cc:1397] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:28:10.640398 4590 raft_consensus.cc:348] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:10.640491 4590 raft_consensus.cc:374] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:10.640529 4590 raft_consensus.cc:729] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d680939a4eaf4486bb37fc854aa43a7a, State: Initialized, Role: FOLLOWER
I20260809 19:28:10.640584 4590 consensus_queue.cc:260] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:10.640679 4590 ts_tablet_manager.cc:1428] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:28:10.641304 5007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.636910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:44955 (local address 127.0.0.1:46758)
0809 19:28:10.637212 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:10.637223 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:10.637271 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:10.637642 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:10.637643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:10.637646 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:10.637684 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.637685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.638900 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.638901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.640010 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.640013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.640982 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.640983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.641118 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.641138 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 19:28:10.641190 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 19:28:10.641231 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":93,"threads_started":1}
I20260809 19:28:10.641324 5008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.636987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46758 (local address 127.21.81.195:44955)
0809 19:28:10.637406 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:10.637410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:10.637418 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:28:10.637449 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:10.637450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:10.637463 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:10.637532 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:10.637769 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.638776 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.640156 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.640903 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.640980 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.641032 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:10.641209 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 19:28:10.641251 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":98,"threads_started":1}
I20260809 19:28:10.641871 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.636794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:38359 (local address 127.0.0.1:57970)
0809 19:28:10.636851 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:10.636862 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:10.636900 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:10.637379 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:10.637380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:10.637382 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:10.637423 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.637424 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.638569 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.638570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.639572 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.639576 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.641633 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.641634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.641718 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.641722 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:28:10.641758 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 19:28:10.641805 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 19:28:10.642225 5009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.637007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57970 (local address 127.21.81.194:38359)
0809 19:28:10.637186 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:10.637191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:10.637203 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:28:10.637247 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:10.637249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:10.637267 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:10.637353 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:10.637505 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.638447 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.639657 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.641512 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.641984 (+ 472us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.642053 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:10.642085 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 19:28:10.642118 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
I20260809 19:28:10.642218 4929 tablet_service.cc:1467] Processing CreateTablet for tablet 192a7c48148941adbc91db9bb58d32cf (DEFAULT_TABLE table=test-table [id=b8c5711604624af6a33674f9a5e7757f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:28:10.642560 4929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 192a7c48148941adbc91db9bb58d32cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:10.642867 4786 tablet_service.cc:1467] Processing CreateTablet for tablet 192a7c48148941adbc91db9bb58d32cf (DEFAULT_TABLE table=test-table [id=b8c5711604624af6a33674f9a5e7757f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:28:10.643172 4786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 192a7c48148941adbc91db9bb58d32cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:10.645421 5012 tablet_bootstrap.cc:492] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: Bootstrap starting.
I20260809 19:28:10.645867 5013 tablet_bootstrap.cc:492] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a: Bootstrap starting.
I20260809 19:28:10.646188 5012 tablet_bootstrap.cc:654] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:10.646482 5013 tablet_bootstrap.cc:654] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:10.646754 5012 log.cc:824] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:10.647043 5013 log.cc:824] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:10.647545 5012 tablet_bootstrap.cc:492] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: No bootstrap required, opened a new log
I20260809 19:28:10.647611 5012 ts_tablet_manager.cc:1397] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:28:10.647816 5013 tablet_bootstrap.cc:492] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a: No bootstrap required, opened a new log
I20260809 19:28:10.647897 5013 ts_tablet_manager.cc:1397] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:28:10.649251 5012 raft_consensus.cc:348] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:10.649377 5012 raft_consensus.cc:374] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:10.649398 5012 raft_consensus.cc:729] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ae7c217444f433cba51e7867df42b13, State: Initialized, Role: FOLLOWER
I20260809 19:28:10.649523 5012 consensus_queue.cc:260] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:10.649557 5013 raft_consensus.cc:348] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:10.649767 5013 raft_consensus.cc:374] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:10.649813 5013 raft_consensus.cc:729] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36410b54d7114a78afdaae0a02d1c95a, State: Initialized, Role: FOLLOWER
I20260809 19:28:10.649847 4999 heartbeater.cc:503] Master 127.21.81.254:33815 was elected leader, sending a full tablet report...
I20260809 19:28:10.649927 5013 consensus_queue.cc:260] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:10.650125 5012 ts_tablet_manager.cc:1428] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:28:10.650219 5013 ts_tablet_manager.cc:1428] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 19:28:10.650229 4856 heartbeater.cc:503] Master 127.21.81.254:33815 was elected leader, sending a full tablet report...
I20260809 19:28:10.681731 5018 raft_consensus.cc:482] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:28:10.681859 5018 raft_consensus.cc:504] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:10.682183 5018 leader_election.cc:288] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ae7c217444f433cba51e7867df42b13 (127.21.81.195:44955), d680939a4eaf4486bb37fc854aa43a7a (127.21.81.193:36703)
I20260809 19:28:10.685319 4858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.682387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:36703 (local address 127.21.81.194:49793)
0809 19:28:10.682435 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:10.682447 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:10.682506 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:10.682691 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:10.682692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:10.682703 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:10.682791 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.682793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.683633 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.683634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.684419 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.684421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.685056 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.685057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.685140 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.685143 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:28:10.685198 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 19:28:10.685227 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 19:28:10.685333 4713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.682465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:49793 (local address 127.21.81.193:36703)
0809 19:28:10.682525 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:10.682527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:10.682537 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:28:10.682567 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:10.682568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:10.682610 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:10.682679 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:10.682880 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.683522 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.684505 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.684971 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.685028 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.685102 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:10.685213 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 19:28:10.685264 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:28:10.685407 5008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.682467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:39753 (local address 127.21.81.195:44955)
0809 19:28:10.682529 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:10.682531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:10.682824 (+ 293us) server_negotiation.cc:407] Connection header received
0809 19:28:10.682958 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:10.682959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:10.682970 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:10.683023 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:10.683189 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.683693 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.684623 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.685025 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.685106 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.685116 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:10.685294 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 19:28:10.685339 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:28:10.685639 5019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.682387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:44955 (local address 127.21.81.194:39753)
0809 19:28:10.682751 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:10.682763 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:10.682851 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:10.683052 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:10.683053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:10.683055 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:10.683119 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.683120 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.683777 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.683778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.684533 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:10.684535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.685121 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.685122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:10.685214 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.685219 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:28:10.685274 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 19:28:10.685309 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":174,"thread_start_us":123,"threads_started":1}
I20260809 19:28:10.685716 4655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "192a7c48148941adbc91db9bb58d32cf" candidate_uuid: "36410b54d7114a78afdaae0a02d1c95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d680939a4eaf4486bb37fc854aa43a7a" is_pre_election: true
I20260809 19:28:10.685917 4655 raft_consensus.cc:2454] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36410b54d7114a78afdaae0a02d1c95a in term 0.
I20260809 19:28:10.686038 4950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "192a7c48148941adbc91db9bb58d32cf" candidate_uuid: "36410b54d7114a78afdaae0a02d1c95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae7c217444f433cba51e7867df42b13" is_pre_election: true
I20260809 19:28:10.686223 4950 raft_consensus.cc:2454] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36410b54d7114a78afdaae0a02d1c95a in term 0.
I20260809 19:28:10.686240 4732 leader_election.cc:302] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36410b54d7114a78afdaae0a02d1c95a, d680939a4eaf4486bb37fc854aa43a7a; no voters:
I20260809 19:28:10.686352 5018 raft_consensus.cc:2781] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:28:10.686441 5018 raft_consensus.cc:482] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:28:10.686478 5018 raft_consensus.cc:3037] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:10.687211 5018 raft_consensus.cc:504] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:10.687378 5018 leader_election.cc:288] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [CANDIDATE]: Term 1 election: Requested vote from peers 4ae7c217444f433cba51e7867df42b13 (127.21.81.195:44955), d680939a4eaf4486bb37fc854aa43a7a (127.21.81.193:36703)
I20260809 19:28:10.687526 4950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "192a7c48148941adbc91db9bb58d32cf" candidate_uuid: "36410b54d7114a78afdaae0a02d1c95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae7c217444f433cba51e7867df42b13"
I20260809 19:28:10.687604 4950 raft_consensus.cc:3037] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:10.688310 4950 raft_consensus.cc:2454] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36410b54d7114a78afdaae0a02d1c95a in term 1.
I20260809 19:28:10.688436 4655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "192a7c48148941adbc91db9bb58d32cf" candidate_uuid: "36410b54d7114a78afdaae0a02d1c95a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d680939a4eaf4486bb37fc854aa43a7a"
I20260809 19:28:10.688494 4734 leader_election.cc:302] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36410b54d7114a78afdaae0a02d1c95a, 4ae7c217444f433cba51e7867df42b13; no voters:
I20260809 19:28:10.688521 4655 raft_consensus.cc:3037] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:10.688608 5018 raft_consensus.cc:2781] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:28:10.688799 5018 raft_consensus.cc:686] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 1 LEADER]: Becoming Leader. State: Replica: 36410b54d7114a78afdaae0a02d1c95a, State: Running, Role: LEADER
I20260809 19:28:10.688902 5018 consensus_queue.cc:237] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:10.689440 4655 raft_consensus.cc:2454] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36410b54d7114a78afdaae0a02d1c95a in term 1.
I20260809 19:28:10.689778 4504 catalog_manager.cc:5495] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a reported cstate change: term changed from 0 to 1, leader changed from <none> to 36410b54d7114a78afdaae0a02d1c95a (127.21.81.194). New cstate: current_term: 1 leader_uuid: "36410b54d7114a78afdaae0a02d1c95a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } health_report { overall_health: UNKNOWN } } }
I20260809 19:28:10.699261 5009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.696938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57984 (local address 127.21.81.194:38359)
0809 19:28:10.696989 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:10.696991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:10.697008 (+ 17us) server_negotiation.cc:407] Connection header received
0809 19:28:10.697078 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:10.697078 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:10.697094 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:10.697178 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:28:10.697314 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.697817 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.698499 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:10.698684 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:10.698757 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:10.698904 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:28:10.698995 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:28:10.699081 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:10.699121 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 19:28:10.699153 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:28:10.700896 4950 raft_consensus.cc:1261] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13 [term 1 FOLLOWER]: Refusing update from remote peer 36410b54d7114a78afdaae0a02d1c95a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:28:10.700886 4655 raft_consensus.cc:1261] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Refusing update from remote peer 36410b54d7114a78afdaae0a02d1c95a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:28:10.701277 5020 consensus_queue.cc:1035] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:28:10.701416 5018 consensus_queue.cc:1035] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:28:10.703408 5023 mvcc.cc:205] Tried to move back new op lower bound from 7316699917108506624 to 7316699917062492160. Current Snapshot: MvccSnapshot[applied={T|T < 7316699917108506624}]
I20260809 19:28:10.703713 5025 mvcc.cc:205] Tried to move back new op lower bound from 7316699917108506624 to 7316699917062492160. Current Snapshot: MvccSnapshot[applied={T|T < 7316699917108506624}]
I20260809 19:28:10.704363 5024 mvcc.cc:205] Tried to move back new op lower bound from 7316699917108506624 to 7316699917062492160. Current Snapshot: MvccSnapshot[applied={T|T < 7316699917108506624 or (T in {7316699917108506624})}]
I20260809 19:28:10.706432 4505 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:35986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:28:10.706535 4505 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:35986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:28:10.707207 4505 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: Sending DeleteTablet for 3 replicas of tablet 192a7c48148941adbc91db9bb58d32cf
I20260809 19:28:10.707545 4635 tablet_service.cc:1514] Processing DeleteTablet for tablet 192a7c48148941adbc91db9bb58d32cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:10 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:60316
I20260809 19:28:10.707581 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 4545
I20260809 19:28:10.707695 4929 tablet_service.cc:1514] Processing DeleteTablet for tablet 192a7c48148941adbc91db9bb58d32cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:10 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:46758
I20260809 19:28:10.707873 4714 tablet_replica.cc:330] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a: stopping tablet replica
I20260809 19:28:10.707926 4714 raft_consensus.cc:2229] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:28:10.707981 4714 raft_consensus.cc:2258] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 19:28:10.715128 4490 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:36703 (error 108)
I20260809 19:28:10.715364 4786 tablet_service.cc:1514] Processing DeleteTablet for tablet 192a7c48148941adbc91db9bb58d32cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:10 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:57970
I20260809 19:28:10.715708 5033 tablet_replica.cc:330] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: stopping tablet replica
W20260809 19:28:10.715744 4712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:28:10.715576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:36703 (local address 127.0.0.1:60326)
0809 19:28:10.715651 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:10.715708 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:36703: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 19:28:10.715833 5033 raft_consensus.cc:2229] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:28:10.715909 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 4709
I20260809 19:28:10.715962 5033 raft_consensus.cc:2258] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 19:28:10.715873 4490 catalog_manager.cc:4584] TS d680939a4eaf4486bb37fc854aa43a7a (127.21.81.193:36703): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 192a7c48148941adbc91db9bb58d32cf: Network error: Client connection negotiation failed: client connection to 127.21.81.193:36703: connect: Connection reset by peer (error 104)
I20260809 19:28:10.717409 5033 ts_tablet_manager.cc:1905] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:28:10.719594 5033 ts_tablet_manager.cc:1918] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:28:10.719691 5033 log.cc:1196] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/192a7c48148941adbc91db9bb58d32cf
I20260809 19:28:10.720131 5033 ts_tablet_manager.cc:1939] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: Deleting consensus metadata
I20260809 19:28:10.720661 4492 catalog_manager.cc:4841] TS 4ae7c217444f433cba51e7867df42b13 (127.21.81.195:44955): tablet 192a7c48148941adbc91db9bb58d32cf (table test-table [id=b8c5711604624af6a33674f9a5e7757f]) successfully deleted
I20260809 19:28:10.722767 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:10.722646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:38359 (local address 127.0.0.1:57998)
0809 19:28:10.722701 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:10.722741 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.194:38359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 19:28:10.722790 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 4859
I20260809 19:28:10.729449 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 4470
I20260809 19:28:10.736644 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:33815
--webserver_interface=127.21.81.254
--webserver_port=33175
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:33815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:10.822346 5036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:10.822572 5036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:10.822635 5036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:10.823812 5036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:28:10.823854 5036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:10.823869 5036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:10.823882 5036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:28:10.823933 5036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:28:10.825414 5036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:33815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:33815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=33175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:10.825698 5036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:10.825959 5036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:10.830952 5041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:10.830945 5042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:10.830981 5036 server_base.cc:1034] running on GCE node
W20260809 19:28:10.830950 5044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:10.831380 5036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:10.831723 5036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:10.832887 5036 hybrid_clock.cc:648] HybridClock initialized: now 1786303690832882 us; error 30 us; skew 500 ppm
Aug 09 19:28:10 dist-test-slave-njxd krb5kdc[3872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:10.838783 5036 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:28:10.839178 5036 webserver.cc:460] Webserver started at http://127.21.81.254:33175/ using document root <none> and password file <none>
I20260809 19:28:10.839425 5036 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:10.839473 5036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:10.840860 5036 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:10.841548 5052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:10.841753 5036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:10.841857 5036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "409dc59891924d03b336f94aa280d3f2"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "6b33480b91a610796dbe1ff76542641a"
server_key_iv: "9c78db8cf574888ef8a705b153610a8c"
server_key_version: "encryptionkey@0"
I20260809 19:28:10.842221 5036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:10.873744 5036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:10.876322 5036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:10.876500 5036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:10.880816 5036 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:33815
I20260809 19:28:10.880877 5108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:33815 every 8 connection(s)
I20260809 19:28:10.881269 5036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:28:10.882848 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 5036
I20260809 19:28:10.882978 5109 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:28:10.883163 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:36703
--local_ip_for_outbound_sockets=127.21.81.193
--tserver_master_addrs=127.21.81.254:33815
--webserver_port=44703
--webserver_interface=127.21.81.193
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:28:10.883764 5109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: Bootstrap starting.
I20260809 19:28:10.892987 5109 log.cc:824] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:10.895362 5109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:28:10.895691 5109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2: Bootstrap complete.
I20260809 19:28:10.898092 5109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } }
I20260809 19:28:10.898306 5109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 409dc59891924d03b336f94aa280d3f2, State: Initialized, Role: FOLLOWER
I20260809 19:28:10.898475 5109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } }
I20260809 19:28:10.898536 5109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:28:10.898583 5109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:28:10.898633 5109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 2 FOLLOWER]: Advancing to term 3
I20260809 19:28:10.899596 5109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } }
I20260809 19:28:10.899704 5109 leader_election.cc:302] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 409dc59891924d03b336f94aa280d3f2; no voters:
I20260809 19:28:10.899870 5109 leader_election.cc:288] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 19:28:10.899945 5113 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 19:28:10.900069 5113 raft_consensus.cc:686] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [term 3 LEADER]: Becoming Leader. State: Replica: 409dc59891924d03b336f94aa280d3f2, State: Running, Role: LEADER
I20260809 19:28:10.900178 5109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:28:10.900166 5113 consensus_queue.cc:237] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } }
I20260809 19:28:10.900415 5114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "409dc59891924d03b336f94aa280d3f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } } }
I20260809 19:28:10.900508 5114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:10.900763 5115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 409dc59891924d03b336f94aa280d3f2. Latest consensus state: current_term: 3 leader_uuid: "409dc59891924d03b336f94aa280d3f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409dc59891924d03b336f94aa280d3f2" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 33815 } } }
I20260809 19:28:10.900877 5115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409dc59891924d03b336f94aa280d3f2 [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:10.900915 5118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:28:10.901814 5118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:28:10.901942 5118 catalog_manager.cc:1259] Loaded cluster ID: 63e0d12e503649d9bd136cdfc03ab9bf
I20260809 19:28:10.901978 5118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 19:28:10.902325 5118 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2eb81209ad google::LogMessage::Fail() at ??:0
@ 0x7f2eb812388a google::LogMessage::SendToLog() at ??:0
@ 0x7f2eb81204de google::LogMessage::Flush() at ??:0
@ 0x7f2eb8121a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2ec00410f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f2ec004a8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2eb9fee0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2eb9fe3954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2ebae476db start_thread at ??:0
@ 0x7f2eb6c1e61f clone at ??:0
W20260809 19:28:10.986559 5111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:10.986831 5111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:10.986876 5111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:10.988051 5111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:10.988127 5111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:10.988171 5111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:28:10.989737 5111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:36703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=44703
--tserver_master_addrs=127.21.81.254:33815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:10.990002 5111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:10.990248 5111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:10.995316 5149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:10.995343 5152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:10.995363 5148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:10.995644 5111 server_base.cc:1034] running on GCE node
I20260809 19:28:10.995847 5111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:10.996150 5111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:10.997347 5111 hybrid_clock.cc:648] HybridClock initialized: now 1786303690997334 us; error 28 us; skew 500 ppm
Aug 09 19:28:10 dist-test-slave-njxd krb5kdc[3872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:11.000203 5111 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:28:11.000572 5111 webserver.cc:460] Webserver started at http://127.21.81.193:44703/ using document root <none> and password file <none>
I20260809 19:28:11.000826 5111 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:11.000897 5111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:11.002357 5111 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:11.003183 5159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:11.003376 5111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:11.003479 5111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "d680939a4eaf4486bb37fc854aa43a7a"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "46d5560b6d092402463220898ed73296"
server_key_iv: "62a54f7c26edd3e8668ebb25de63818d"
server_key_version: "encryptionkey@0"
I20260809 19:28:11.003824 5111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:11.013355 5111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:11.015733 5111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:11.015894 5111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:11.016152 5111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:11.016609 5166 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:28:11.018599 5111 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:28:11.018659 5111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:28:11.018724 5111 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:28:11.019372 5111 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:28:11.019440 5111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 19:28:11.019519 5166 tablet_bootstrap.cc:492] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a: Bootstrap starting.
I20260809 19:28:11.026791 5111 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:36703
I20260809 19:28:11.026882 5283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:36703 every 8 connection(s)
I20260809 19:28:11.027231 5111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:28:11.030759 5166 log.cc:824] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:11.032886 5166 tablet_bootstrap.cc:492] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:28:11.032949 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 5111
I20260809 19:28:11.033211 5166 tablet_bootstrap.cc:492] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a: Bootstrap complete.
I20260809 19:28:11.033238 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:38359
--local_ip_for_outbound_sockets=127.21.81.194
--tserver_master_addrs=127.21.81.254:33815
--webserver_port=41671
--webserver_interface=127.21.81.194
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:28:11.034495 5166 ts_tablet_manager.cc:1397] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.006s
I20260809 19:28:11.035758 5166 raft_consensus.cc:348] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:11.036020 5166 raft_consensus.cc:729] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d680939a4eaf4486bb37fc854aa43a7a, State: Initialized, Role: FOLLOWER
I20260809 19:28:11.036192 5166 consensus_queue.cc:260] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:11.036507 5166 ts_tablet_manager.cc:1428] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260809 19:28:11.120725 5289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:11.120916 5289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:11.120940 5289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:11.122201 5289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:11.122243 5289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:11.122269 5289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:28:11.123873 5289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:38359
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=41671
--tserver_master_addrs=127.21.81.254:33815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:11.124118 5289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:11.124357 5289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:11.129395 5297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:11.129403 5296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:11.129660 5289 server_base.cc:1034] running on GCE node
W20260809 19:28:11.129403 5299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:11.129963 5289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:11.130267 5289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:11.131446 5289 hybrid_clock.cc:648] HybridClock initialized: now 1786303691131428 us; error 37 us; skew 500 ppm
Aug 09 19:28:11 dist-test-slave-njxd krb5kdc[3872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303691, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:11.134156 5289 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:28:11.134658 5289 webserver.cc:460] Webserver started at http://127.21.81.194:41671/ using document root <none> and password file <none>
I20260809 19:28:11.134887 5289 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:11.134955 5289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:11.136406 5289 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 19:28:11.137204 5306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:11.137408 5289 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:11.137511 5289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "36410b54d7114a78afdaae0a02d1c95a"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "00d91be06c7b060dc6f0d1f4a2ffee84"
server_key_iv: "9514397819128cb3884446b393152076"
server_key_version: "encryptionkey@0"
I20260809 19:28:11.137924 5289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:11.148144 5289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:11.150514 5289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:11.150683 5289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:11.150920 5289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:11.151345 5313 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:28:11.153151 5289 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:28:11.153215 5289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:28:11.153275 5289 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:28:11.154368 5289 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:28:11.154425 5289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 19:28:11.154484 5313 tablet_bootstrap.cc:492] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a: Bootstrap starting.
I20260809 19:28:11.161429 5289 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:38359
I20260809 19:28:11.161517 5430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:38359 every 8 connection(s)
I20260809 19:28:11.161839 5289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:28:11.165654 5313 log.cc:824] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:11.167853 5313 tablet_bootstrap.cc:492] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:28:11.168151 5313 tablet_bootstrap.cc:492] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a: Bootstrap complete.
I20260809 19:28:11.169552 5313 ts_tablet_manager.cc:1397] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260809 19:28:11.170040 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 5289
I20260809 19:28:11.170322 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:44955
--local_ip_for_outbound_sockets=127.21.81.195
--tserver_master_addrs=127.21.81.254:33815
--webserver_port=41123
--webserver_interface=127.21.81.195
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:35613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:28:11.170883 5313 raft_consensus.cc:348] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:11.171131 5313 raft_consensus.cc:729] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36410b54d7114a78afdaae0a02d1c95a, State: Initialized, Role: FOLLOWER
I20260809 19:28:11.171283 5313 consensus_queue.cc:260] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:11.171577 5313 ts_tablet_manager.cc:1428] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
W20260809 19:28:11.255112 5438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:11.255359 5438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:11.255401 5438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:11.257364 5438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:11.257426 5438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:11.257474 5438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:28:11.260474 5438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:35613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:44955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=41123
--tserver_master_addrs=127.21.81.254:33815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:11.260757 5438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:11.261023 5438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:11.262715 5433 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:28:11.162523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:48717)
0809 19:28:11.162714 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:11.162767 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:11.163609 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:11.262518 (+ 98909us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
W20260809 19:28:11.262720 5287 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:28:11.028047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:55635)
0809 19:28:11.028258 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:11.028316 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:11.029085 (+ 769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:11.262519 (+233434us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":85,"threads_started":1}
W20260809 19:28:11.262976 5431 heartbeater.cc:644] Failed to heartbeat to 127.21.81.254:33815 (0 consecutive failures): Network error: Failed to ping master at 127.21.81.254:33815: Client connection negotiation failed: client connection to 127.21.81.254:33815: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 19:28:11.262975 5284 heartbeater.cc:644] Failed to heartbeat to 127.21.81.254:33815 (0 consecutive failures): Network error: Failed to ping master at 127.21.81.254:33815: Client connection negotiation failed: client connection to 127.21.81.254:33815: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 19:28:11.263285 5433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:11.263170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:58483)
0809 19:28:11.263228 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:11.263266 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 19:28:11.263283 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:11.263170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:46855)
0809 19:28:11.263216 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:11.263261 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 19:28:11.263558 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:11.263482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:54241)
0809 19:28:11.263513 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:11.263542 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 19:28:11.263605 5433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:11.263472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:37249)
0809 19:28:11.263537 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:11.263575 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 19:28:11.263679 5284 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:28:11.263728 5431 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:28:11.266947 5447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:11.266964 5438 server_base.cc:1034] running on GCE node
W20260809 19:28:11.266981 5444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:11.267069 5443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:11.267391 5438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:11.267647 5438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:11.268818 5438 hybrid_clock.cc:648] HybridClock initialized: now 1786303691268799 us; error 46 us; skew 500 ppm
Aug 09 19:28:11 dist-test-slave-njxd krb5kdc[3872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303691, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:11.271241 5438 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:28:11.271667 5438 webserver.cc:460] Webserver started at http://127.21.81.195:41123/ using document root <none> and password file <none>
I20260809 19:28:11.271903 5438 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:11.271974 5438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:11.273360 5438 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:11.274072 5454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:11.274238 5438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:11.274335 5438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "4ae7c217444f433cba51e7867df42b13"
format_stamp: "Formatted at 2026-08-09 19:28:09 on dist-test-slave-njxd"
server_key: "ebafe32b1d049dada95916fd43afec84"
server_key_iv: "c9e2a1b36fabed894b1290e20d0e0195"
server_key_version: "encryptionkey@0"
I20260809 19:28:11.274695 5438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:11.293489 5438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:11.295882 5438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:11.296048 5438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:11.296296 5438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:11.296604 5438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:11.296658 5438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:11.296701 5438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:11.296726 5438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:11.302783 5438 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:44955
I20260809 19:28:11.302896 5581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:44955 every 8 connection(s)
I20260809 19:28:11.303215 5438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:28:11.304306 5584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:11.303919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:50107)
0809 19:28:11.304079 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:11.304156 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
W20260809 19:28:11.304716 5582 heartbeater.cc:644] Failed to heartbeat to 127.21.81.254:33815 (0 consecutive failures): Network error: Failed to ping master at 127.21.81.254:33815: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
I20260809 19:28:11.305003 5584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:11.304918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:53375)
0809 19:28:11.304963 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:11.304988 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 19:28:11.305241 5584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:11.305195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:59205)
0809 19:28:11.305214 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:11.305231 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260809 19:28:11.305361 5582 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 19:28:11.306541 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 5438
I20260809 19:28:12.264333 5587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:12.264074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:46175)
0809 19:28:12.264234 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:12.264274 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":51,"threads_started":1}
I20260809 19:28:12.264357 5586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:12.264074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:53617)
0809 19:28:12.264234 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:12.264274 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
I20260809 19:28:12.305931 5589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:12.305727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:58425)
0809 19:28:12.305851 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:12.305881 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260809 19:28:12.412230 5590 raft_consensus.cc:482] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:28:12.412400 5590 raft_consensus.cc:504] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:12.412711 5590 leader_election.cc:288] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 36410b54d7114a78afdaae0a02d1c95a (127.21.81.194:38359), 4ae7c217444f433cba51e7867df42b13 (127.21.81.195:44955)
Aug 09 19:28:12 dist-test-slave-njxd krb5kdc[3872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.195@KRBTEST.COM
Aug 09 19:28:12 dist-test-slave-njxd krb5kdc[3872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.194@KRBTEST.COM
I20260809 19:28:12.422489 5591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:12.412997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:44955 (local address 127.21.81.193:57347)
0809 19:28:12.413098 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:12.413107 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:12.413192 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:12.414069 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:12.414070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:12.414254 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:12.414294 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:12.414296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:12.415356 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:12.415357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:12.415952 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:12.415954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:12.416530 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:12.416531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:12.416660 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:12.417230 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:12.417237 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:12.418817 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:12.420957 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:12.420963 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:12.420964 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:12.421172 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:12.421447 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:12.421448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:12.421448 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:12.421543 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:12.422020 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:12.422026 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:12.422185 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 19:28:12.422296 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 19:28:12.422415 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":22,"thread_start_us":48,"threads_started":1}
I20260809 19:28:12.422628 5593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:12.413107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:57347 (local address 127.21.81.195:44955)
0809 19:28:12.413274 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:12.413284 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:12.413317 (+ 33us) server_negotiation.cc:407] Connection header received
0809 19:28:12.413883 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:12.413886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:12.413929 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:12.414079 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:12.414405 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:12.415253 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:12.416038 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:12.416436 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:12.416557 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:12.418940 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:12.418943 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:12.418944 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:12.418952 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:12.420867 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:12.421256 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:12.421259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:12.421259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:12.421372 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:12.421659 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:12.421660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:12.421660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:12.421944 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:12.422033 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:12.422366 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 19:28:12.422495 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":55,"threads_started":1}
I20260809 19:28:12.422896 5586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:12.412891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:38359 (local address 127.21.81.193:60413)
0809 19:28:12.412961 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:12.412976 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:12.413057 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:12.413481 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:12.413483 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:12.413833 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:12.414017 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:12.414023 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:12.415155 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:12.415156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:12.415951 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:12.415953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:12.416539 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:12.416540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:12.416661 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:12.417297 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:12.417302 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:12.419244 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:12.421670 (+ 2426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:12.421671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:12.421671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:12.421834 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:12.422096 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:12.422097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:12.422097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:12.422161 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:12.422661 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:12.422663 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:12.422708 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 19:28:12.422753 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:28:12.422835 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":21}
I20260809 19:28:12.423342 5592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:12.413025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:60413 (local address 127.21.81.194:38359)
0809 19:28:12.413179 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:12.413185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:12.413196 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:28:12.413299 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:12.413301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:12.413327 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:12.413417 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:12.414131 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:12.415026 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:12.416041 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:12.416445 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:12.416558 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:12.419607 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:12.419610 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:12.419610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:12.419620 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:12.421555 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:12.421913 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:12.421915 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:12.421916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:12.422028 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:12.422231 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:12.422232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:12.422232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:12.422536 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:12.423053 (+ 517us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:12.423137 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 19:28:12.423252 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":102,"threads_started":1}
I20260809 19:28:12.423717 5380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "192a7c48148941adbc91db9bb58d32cf" candidate_uuid: "d680939a4eaf4486bb37fc854aa43a7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "36410b54d7114a78afdaae0a02d1c95a" is_pre_election: true
W20260809 19:28:12.423748 5163 leader_election.cc:341] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4ae7c217444f433cba51e7867df42b13 (127.21.81.195:44955): Not found: Tablet not found: 192a7c48148941adbc91db9bb58d32cf
I20260809 19:28:12.423837 5380 raft_consensus.cc:2454] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d680939a4eaf4486bb37fc854aa43a7a in term 1.
I20260809 19:28:12.424103 5163 leader_election.cc:302] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36410b54d7114a78afdaae0a02d1c95a, d680939a4eaf4486bb37fc854aa43a7a; no voters: 4ae7c217444f433cba51e7867df42b13
I20260809 19:28:12.424219 5590 raft_consensus.cc:2781] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 19:28:12.424281 5590 raft_consensus.cc:482] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:28:12.424306 5590 raft_consensus.cc:3037] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:28:12.425387 5590 raft_consensus.cc:504] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:12.425526 5590 leader_election.cc:288] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [CANDIDATE]: Term 2 election: Requested vote from peers 36410b54d7114a78afdaae0a02d1c95a (127.21.81.194:38359), 4ae7c217444f433cba51e7867df42b13 (127.21.81.195:44955)
I20260809 19:28:12.425693 5380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "192a7c48148941adbc91db9bb58d32cf" candidate_uuid: "d680939a4eaf4486bb37fc854aa43a7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "36410b54d7114a78afdaae0a02d1c95a"
I20260809 19:28:12.425765 5380 raft_consensus.cc:3037] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 1 FOLLOWER]: Advancing to term 2
W20260809 19:28:12.425845 5163 leader_election.cc:341] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 4ae7c217444f433cba51e7867df42b13 (127.21.81.195:44955): Not found: Tablet not found: 192a7c48148941adbc91db9bb58d32cf
I20260809 19:28:12.426832 5380 raft_consensus.cc:2454] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d680939a4eaf4486bb37fc854aa43a7a in term 2.
I20260809 19:28:12.426991 5163 leader_election.cc:302] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36410b54d7114a78afdaae0a02d1c95a, d680939a4eaf4486bb37fc854aa43a7a; no voters: 4ae7c217444f433cba51e7867df42b13
I20260809 19:28:12.427099 5590 raft_consensus.cc:2781] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:28:12.427253 5590 raft_consensus.cc:686] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [term 2 LEADER]: Becoming Leader. State: Replica: d680939a4eaf4486bb37fc854aa43a7a, State: Running, Role: LEADER
I20260809 19:28:12.427357 5590 consensus_queue.cc:237] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:12.427603 5586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:12.427511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:57663)
0809 19:28:12.427555 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:12.427577 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 19:28:12.953227 5380 raft_consensus.cc:1261] T 192a7c48148941adbc91db9bb58d32cf P 36410b54d7114a78afdaae0a02d1c95a [term 2 FOLLOWER]: Refusing update from remote peer d680939a4eaf4486bb37fc854aa43a7a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 19:28:12.953972 5596 consensus_queue.cc:1035] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 19:28:12.955637 5599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:12.955308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:58677)
0809 19:28:12.955499 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:12.955572 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":123,"threads_started":1}
I20260809 19:28:12.957197 5599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:12.956997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:37597)
0809 19:28:12.957072 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:12.957120 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 19:28:12.957697 5163 consensus_peers.cc:487] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a -> Peer 4ae7c217444f433cba51e7867df42b13 (127.21.81.195:44955): Couldn't send request to peer 4ae7c217444f433cba51e7867df42b13. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 192a7c48148941adbc91db9bb58d32cf. This is attempt 1: this message will repeat every 5th retry.
I20260809 19:28:12.957861 5605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:12.957367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:35693)
0809 19:28:12.957628 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:12.957700 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":158,"threads_started":1}
I20260809 19:28:13.307014 5606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:13.306592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:41505)
0809 19:28:13.306865 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:13.306930 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":157,"threads_started":1}
I20260809 19:28:13.475370 5608 ts_tablet_manager.cc:927] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: Initiating tablet copy from peer d680939a4eaf4486bb37fc854aa43a7a (127.21.81.193:36703)
I20260809 19:28:13.475739 5608 tablet_copy_client.cc:316] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: tablet copy: Beginning tablet copy session from remote peer at address 127.21.81.193:36703
Aug 09 19:28:13 dist-test-slave-njxd krb5kdc[3872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303691, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.193@KRBTEST.COM
I20260809 19:28:13.487758 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:13.476347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:54275 (local address 127.21.81.193:36703)
0809 19:28:13.476641 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:13.476648 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:13.476667 (+ 19us) server_negotiation.cc:407] Connection header received
0809 19:28:13.476712 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:13.476715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:13.476738 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:13.476832 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:13.477498 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:13.478575 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:13.479618 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:13.480050 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:13.480133 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:13.483540 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:13.483557 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:13.483557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:13.483565 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:13.485973 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:13.486539 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:13.486547 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:13.486547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:13.486647 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:13.486962 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:13.486963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:13.486964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:13.487220 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:13.487330 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:13.487536 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 19:28:13.487658 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":120,"threads_started":1}
I20260809 19:28:13.487775 5606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:13.476218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:36703 (local address 127.21.81.195:54275)
0809 19:28:13.476329 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:13.476360 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:13.476563 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:13.476843 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:13.476847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:13.477294 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:13.477402 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:13.477404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:13.478689 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:13.478690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:13.479516 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:13.479519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:13.480218 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:13.480220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:13.480325 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:13.481187 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:13.481193 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:13.483344 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:13.486076 (+ 2732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:13.486090 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:13.486093 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:13.486437 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:13.486722 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:13.486723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:13.486723 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:13.486841 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:13.487299 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:13.487309 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:13.487382 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 19:28:13.487483 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 19:28:13.487648 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 19:28:13.488725 5255 tablet_copy_service.cc:140] P d680939a4eaf4486bb37fc854aa43a7a: Received BeginTabletCopySession request for tablet 192a7c48148941adbc91db9bb58d32cf from peer 4ae7c217444f433cba51e7867df42b13 ({username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:54275)
I20260809 19:28:13.488850 5255 tablet_copy_service.cc:161] P d680939a4eaf4486bb37fc854aa43a7a: Beginning new tablet copy session on tablet 192a7c48148941adbc91db9bb58d32cf from peer 4ae7c217444f433cba51e7867df42b13 at {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:54275: session id = 4ae7c217444f433cba51e7867df42b13-192a7c48148941adbc91db9bb58d32cf
I20260809 19:28:13.489684 5255 tablet_copy_source_session.cc:205] T 192a7c48148941adbc91db9bb58d32cf P d680939a4eaf4486bb37fc854aa43a7a: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 19:28:13.491093 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 192a7c48148941adbc91db9bb58d32cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:13.495535 5608 tablet_copy_client.cc:791] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: tablet copy: Starting download of 0 data blocks...
I20260809 19:28:13.495827 5608 tablet_copy_client.cc:655] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: tablet copy: Starting download of 1 WAL segments...
I20260809 19:28:13.497063 5608 tablet_copy_client.cc:531] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 19:28:13.498188 5608 tablet_bootstrap.cc:492] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: Bootstrap starting.
I20260809 19:28:13.500422 5608 log.cc:824] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:13.504160 5608 tablet_bootstrap.cc:492] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:28:13.504540 5608 tablet_bootstrap.cc:492] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: Bootstrap complete.
I20260809 19:28:13.504769 5608 ts_tablet_manager.cc:1397] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.001s
I20260809 19:28:13.506498 5608 raft_consensus.cc:348] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:13.506888 5608 raft_consensus.cc:729] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ae7c217444f433cba51e7867df42b13, State: Initialized, Role: FOLLOWER
I20260809 19:28:13.507128 5608 consensus_queue.cc:260] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36410b54d7114a78afdaae0a02d1c95a" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 38359 } } peers { permanent_uuid: "4ae7c217444f433cba51e7867df42b13" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44955 } } peers { permanent_uuid: "d680939a4eaf4486bb37fc854aa43a7a" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 36703 } }
I20260809 19:28:13.507534 5608 ts_tablet_manager.cc:1428] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13: Time spent starting tablet: real 0.003s user 0.002s sys 0.003s
I20260809 19:28:13.507972 5606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:13.507842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:37963)
0809 19:28:13.507917 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:13.507948 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 19:28:13.507987 5255 tablet_copy_service.cc:342] P d680939a4eaf4486bb37fc854aa43a7a: Request end of tablet copy session 4ae7c217444f433cba51e7867df42b13-192a7c48148941adbc91db9bb58d32cf received from {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:54275
I20260809 19:28:13.508057 5255 tablet_copy_service.cc:433] P d680939a4eaf4486bb37fc854aa43a7a: ending tablet copy session 4ae7c217444f433cba51e7867df42b13-192a7c48148941adbc91db9bb58d32cf on tablet 192a7c48148941adbc91db9bb58d32cf with peer 4ae7c217444f433cba51e7867df42b13
I20260809 19:28:13.508455 5606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:13.508335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:46209)
0809 19:28:13.508384 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:13.508424 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 19:28:13.958096 5614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:13.957825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:50063)
0809 19:28:13.957982 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:13.958035 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260809 19:28:13.958662 5615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:13.958399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:34029)
0809 19:28:13.958563 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:13.958614 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":110,"threads_started":1}
I20260809 19:28:14.066277 5526 raft_consensus.cc:1203] T 192a7c48148941adbc91db9bb58d32cf P 4ae7c217444f433cba51e7867df42b13 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260809 19:28:14.066391 5615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:14.066209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:56129)
0809 19:28:14.066286 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:14.066334 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 19:28:14.067133 5618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:14.066859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:55945)
0809 19:28:14.067000 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:14.067057 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260809 19:28:14.958925 5619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:14.958704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:56879)
0809 19:28:14.958845 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:14.958878 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":66,"threads_started":1}
I20260809 19:28:15.067123 5620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:15.066892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:36613)
0809 19:28:15.067037 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:15.067075 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":67,"threads_started":1}
I20260809 19:28:15.067755 5623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:15.067561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:51217)
0809 19:28:15.067677 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:15.067716 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
I20260809 19:28:15.959702 5624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:15.959475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:34153)
0809 19:28:15.959615 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:15.959654 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260809 19:28:16.067849 5625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:16.067619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:35609)
0809 19:28:16.067768 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:16.067801 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260809 19:28:16.068223 5625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:16.068127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:46827)
0809 19:28:16.068173 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:16.068204 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 19:28:16.068373 5626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:16.068133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:43161)
0809 19:28:16.068264 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:16.068317 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260809 19:28:16.960469 5628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:16.960139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:48317)
0809 19:28:16.960381 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:16.960422 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":81,"threads_started":1}
I20260809 19:28:17.069000 5629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:17.068727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:55185)
0809 19:28:17.068888 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:17.068950 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
I20260809 19:28:17.069060 5630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:17.068823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:47909)
0809 19:28:17.068965 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:17.068999 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":50,"threads_started":1}
I20260809 19:28:17.961241 5631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:17.961027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:53817)
0809 19:28:17.961163 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:17.961195 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260809 19:28:18.069825 5632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:18.069598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:40583)
0809 19:28:18.069735 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:18.069777 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260809 19:28:18.069837 5633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:18.069598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:43163)
0809 19:28:18.069763 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:18.069795 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
I20260809 19:28:18.961961 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:18.961732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:60735)
0809 19:28:18.961871 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:18.961911 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260809 19:28:19.070505 5637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:19.070278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:45789)
0809 19:28:19.070410 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:19.070450 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260809 19:28:19.070505 5636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:19.070261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:52609)
0809 19:28:19.070420 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:19.070450 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":81,"threads_started":1}
I20260809 19:28:19.962700 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:19.962462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:42943)
0809 19:28:19.962602 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:19.962639 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
I20260809 19:28:20.071151 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:20.070927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:56891)
0809 19:28:20.071071 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:20.071103 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260809 19:28:20.071172 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:20.070953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:37073)
0809 19:28:20.071089 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:20.071131 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260809 19:28:20.963375 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:20.963158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:43917)
0809 19:28:20.963287 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:20.963326 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260809 19:28:21.071907 5645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:21.071671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:39793)
0809 19:28:21.071817 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:21.071862 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
I20260809 19:28:21.071907 5644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:21.071662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:46147)
0809 19:28:21.071792 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:21.071844 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260809 19:28:21.964188 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:21.963948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:58247)
0809 19:28:21.964096 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:21.964139 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":63,"threads_started":1}
I20260809 19:28:22.072770 5649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:22.072497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:46145)
0809 19:28:22.072684 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:22.072720 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":98,"threads_started":1}
I20260809 19:28:22.072923 5648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:22.072487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:60855)
0809 19:28:22.072815 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:22.072872 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":99,"threads_started":1}
I20260809 19:28:22.965056 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:22.964792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:40097)
0809 19:28:22.964958 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:22.965007 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":64,"threads_started":1}
I20260809 19:28:23.073460 5651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:23.073208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:48051)
0809 19:28:23.073344 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:23.073392 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260809 19:28:23.073526 5652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:23.073307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:49973)
0809 19:28:23.073447 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:23.073480 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
I20260809 19:28:23.965799 5653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:23.965541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:47663)
0809 19:28:23.965721 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:23.965754 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":125,"threads_started":1}
I20260809 19:28:24.074224 5654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:24.073968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:43711)
0809 19:28:24.074140 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:24.074174 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260809 19:28:24.074406 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:24.074144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:36593)
0809 19:28:24.074297 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:24.074350 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
I20260809 19:28:24.966485 5658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:24.966260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:57797)
0809 19:28:24.966394 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:24.966436 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260809 19:28:25.075269 5660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:25.074985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:50135)
0809 19:28:25.075153 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:25.075208 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":76,"threads_started":1}
I20260809 19:28:25.075292 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:25.074980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:51679)
0809 19:28:25.075160 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:25.075235 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":77,"threads_started":1}
I20260809 19:28:25.967411 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:25.967132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.194:41257)
0809 19:28:25.967319 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:25.967362 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":108,"threads_started":1}
I20260809 19:28:26.076226 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:26.075956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.193:52487)
0809 19:28:26.076120 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:26.076168 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260809 19:28:26.076248 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:26.075956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:33815 (local address 127.21.81.195:50885)
0809 19:28:26.076170 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:26.076206 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":95,"threads_started":1}
I20260809 19:28:26.317757 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 5111
I20260809 19:28:26.325553 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 5289
I20260809 19:28:26.331108 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 5438
2026-08-09T19:28:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18903 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:28:27 dist-test-slave-njxd krb5kdc[5667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:28:27 dist-test-slave-njxd krb5kdc[5667](info): set up 2 sockets
Aug 09 19:28:27 dist-test-slave-njxd krb5kdc[5667](info): commencing operation
krb5kdc: starting...
W20260809 19:28:28.040748 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:28:28 dist-test-slave-njxd krb5kdc[5667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:28:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:28:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:28:28.161459 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39717
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:39717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:28.241408 5683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:28.241629 5683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:28.241654 5683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:28.242766 5683 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 19:28:28.242800 5683 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 19:28:28.242813 5683 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 19:28:28.242825 5683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:28:28.242837 5683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:28.242851 5683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:28.242862 5683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:28:28.242874 5683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:28:28.244284 5683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:39717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:28.244477 5683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:28.244644 5683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:28:28.249886 5683 server_base.cc:1034] running on GCE node
W20260809 19:28:28.249840 5689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:28.249851 5690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:28.249931 5692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:28.250370 5683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:28.250665 5683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:28.251817 5683 hybrid_clock.cc:648] HybridClock initialized: now 1786303708251801 us; error 28 us; skew 500 ppm
Aug 09 19:28:28 dist-test-slave-njxd krb5kdc[5667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303708, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:28.254366 5683 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:28:28.254824 5683 webserver.cc:460] Webserver started at http://127.21.81.254:46309/ using document root <none> and password file <none>
I20260809 19:28:28.255053 5683 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:28.255105 5683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:28.255268 5683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:28:28.256371 5683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "b13d62f274a24b66923d5daaf5ba7b9a"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "102d37588b3fcc67bb4c84b0f3887f1f"
server_key_iv: "9e7e279984ed654778d2907f121f7419"
server_key_version: "encryptionkey@0"
I20260809 19:28:28.256762 5683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "b13d62f274a24b66923d5daaf5ba7b9a"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "102d37588b3fcc67bb4c84b0f3887f1f"
server_key_iv: "9e7e279984ed654778d2907f121f7419"
server_key_version: "encryptionkey@0"
I20260809 19:28:28.258260 5683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260809 19:28:28.259030 5700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:28.259212 5683 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:28:28.259315 5683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "b13d62f274a24b66923d5daaf5ba7b9a"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "102d37588b3fcc67bb4c84b0f3887f1f"
server_key_iv: "9e7e279984ed654778d2907f121f7419"
server_key_version: "encryptionkey@0"
I20260809 19:28:28.259395 5683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:28.267879 5683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:28.269706 5683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:28.269856 5683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:28.274169 5683 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:39717
I20260809 19:28:28.274204 5757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:39717 every 8 connection(s)
I20260809 19:28:28.274541 5683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:28:28.275475 5758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:28.276024 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 5683
I20260809 19:28:28.276144 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:28:28.276332 21831 external_mini_cluster.cc:1428] Setting key 3a071d72a115e64d9166ae9ad9a25535
I20260809 19:28:28.278644 5758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Bootstrap starting.
I20260809 19:28:28.279258 5758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:28.279557 5758 log.cc:824] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:28.280333 5758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: No bootstrap required, opened a new log
I20260809 19:28:28.282958 5758 raft_consensus.cc:348] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } }
I20260809 19:28:28.283087 5758 raft_consensus.cc:374] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:28.283123 5758 raft_consensus.cc:729] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13d62f274a24b66923d5daaf5ba7b9a, State: Initialized, Role: FOLLOWER
I20260809 19:28:28.283259 5758 consensus_queue.cc:260] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } }
I20260809 19:28:28.283332 5758 raft_consensus.cc:388] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:28:28.283355 5758 raft_consensus.cc:482] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:28:28.283422 5758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 0 FOLLOWER]: Advancing to term 1
Aug 09 19:28:28 dist-test-slave-njxd krb5kdc[5667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:28.284071 5758 raft_consensus.cc:504] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } }
I20260809 19:28:28.284219 5758 leader_election.cc:302] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b13d62f274a24b66923d5daaf5ba7b9a; no voters:
I20260809 19:28:28.284387 5758 leader_election.cc:288] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:28:28.284454 5763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:28:28.284652 5763 raft_consensus.cc:686] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 1 LEADER]: Becoming Leader. State: Replica: b13d62f274a24b66923d5daaf5ba7b9a, State: Running, Role: LEADER
I20260809 19:28:28.284719 5758 sys_catalog.cc:564] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:28:28.284796 5763 consensus_queue.cc:237] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } }
I20260809 19:28:28.286392 5765 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } } }
I20260809 19:28:28.286593 5763 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b13d62f274a24b66923d5daaf5ba7b9a. Latest consensus state: current_term: 1 leader_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } } }
I20260809 19:28:28.286564 5765 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:28.286808 5763 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [sys.catalog]: This master's current role is: LEADER
W20260809 19:28:28.287221 5778 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:28:28.287276 5778 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:28:28.287326 5780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:28:28.287493 5780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:28:28.287945 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.276872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.21.81.254:39717)
0809 19:28:28.277047 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:28.277054 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:28.277077 (+ 23us) server_negotiation.cc:407] Connection header received
0809 19:28:28.277449 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:28.277455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:28.277496 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:28.277681 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:28.278601 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.279550 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.280630 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.281005 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.281083 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.284036 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:28.284040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:28.284040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:28.284053 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:28.286277 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:28.286703 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:28.286706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:28.286707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:28.286815 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:28.287051 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:28.287052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:28.287053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:28.287360 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:28.287566 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:28.287650 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 19:28:28.287779 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260809 19:28:28.289021 5780 catalog_manager.cc:1347] Generated new cluster ID: 29735a69af56418688e2234ced76f6d5
I20260809 19:28:28.289077 5780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:28:28.297642 5780 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:28:28.298130 5780 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:28:28.302881 5780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Generated new TSK 0
I20260809 19:28:28.303041 5780 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:28:28.334317 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39717
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:28.417555 5785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:28.417769 5785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:28.417793 5785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:28.418998 5785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:28.419054 5785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:28.419080 5785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:28:28.420571 5785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:28.420795 5785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:28.421013 5785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:28.426025 5794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:28.426108 5785 server_base.cc:1034] running on GCE node
W20260809 19:28:28.426136 5790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:28.426074 5791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:28.426419 5785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:28.426622 5785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:28.427806 5785 hybrid_clock.cc:648] HybridClock initialized: now 1786303708427798 us; error 27 us; skew 500 ppm
Aug 09 19:28:28 dist-test-slave-njxd krb5kdc[5667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303708, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:28.430495 5785 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:28:28.430922 5785 webserver.cc:460] Webserver started at http://127.21.81.193:40363/ using document root <none> and password file <none>
I20260809 19:28:28.431155 5785 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:28.431208 5785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:28.431391 5785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:28:28.432443 5785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "783324c00a734f879546e2d6421d5023"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "33e9ad57d392ac2473afab2a1dda6e2b"
server_key_iv: "1b015aea426213b1c97b5db38c0658f4"
server_key_version: "encryptionkey@0"
I20260809 19:28:28.432796 5785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "783324c00a734f879546e2d6421d5023"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "33e9ad57d392ac2473afab2a1dda6e2b"
server_key_iv: "1b015aea426213b1c97b5db38c0658f4"
server_key_version: "encryptionkey@0"
I20260809 19:28:28.434211 5785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:28:28.434957 5801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:28.435143 5785 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:28.435245 5785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "783324c00a734f879546e2d6421d5023"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "33e9ad57d392ac2473afab2a1dda6e2b"
server_key_iv: "1b015aea426213b1c97b5db38c0658f4"
server_key_version: "encryptionkey@0"
I20260809 19:28:28.435320 5785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:28.446063 5785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:28.448432 5785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:28.448604 5785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:28.448848 5785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:28.449182 5785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:28.449255 5785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:28.449299 5785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:28.449327 5785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:28.455667 5785 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:46599
I20260809 19:28:28.455750 5929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:46599 every 8 connection(s)
I20260809 19:28:28.456095 5785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:28:28.459357 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 5785
I20260809 19:28:28.459486 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:28:28.459697 21831 external_mini_cluster.cc:1428] Setting key 19c3877df9b8860e5985810037f04401
Aug 09 19:28:28 dist-test-slave-njxd krb5kdc[5667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303708, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:28.466524 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.457010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:50259 (local address 127.21.81.254:39717)
0809 19:28:28.457077 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:28.457080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:28.457273 (+ 193us) server_negotiation.cc:407] Connection header received
0809 19:28:28.457858 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:28.457860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:28.457871 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:28.457951 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:28.459087 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.459718 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.460780 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.461000 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.461081 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.463499 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:28.463500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:28.463501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:28.463509 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:28.465010 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:28.465451 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:28.465452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:28.465453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:28.465516 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:28.465814 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:28.465815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:28.465815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:28.465937 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:28.466013 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:28.466324 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 19:28:28.466426 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:28:28.466712 5932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.456947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:50259)
0809 19:28:28.457116 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:28.457160 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:28.457733 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:28.457998 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:28.458001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:28.458770 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:28.458979 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.458984 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.459847 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.459849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.460667 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.460669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.461099 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.461101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.461225 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.461839 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:28.461848 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:28.463384 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:28.465112 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:28.465121 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:28.465124 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:28.465366 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:28.465598 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:28.465598 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:28.465599 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:28.465730 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:28.466016 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:28.466024 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:28.466177 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 19:28:28.466287 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 19:28:28.466462 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260809 19:28:28.467284 5930 heartbeater.cc:347] Connected to a master server at 127.21.81.254:39717
I20260809 19:28:28.467375 5930 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:28.467578 5930 heartbeater.cc:511] Master 127.21.81.254:39717 requested a full tablet report, sending...
I20260809 19:28:28.468214 5719 ts_manager.cc:194] Registered new tserver with Master: 783324c00a734f879546e2d6421d5023 (127.21.81.193:46599)
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
I20260809 19:28:28.469226 5719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:50259
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:28:28.488432 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39717
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:28.571489 5935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:28.571740 5935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:28.571779 5935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:28.572912 5935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:28.572983 5935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:28.573028 5935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:28:28.574515 5935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:28.574821 5935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:28.575055 5935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:28.580259 5943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:28.580243 5941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:28.580293 5935 server_base.cc:1034] running on GCE node
W20260809 19:28:28.580261 5940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:28.580651 5935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:28.580852 5935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:28.581985 5935 hybrid_clock.cc:648] HybridClock initialized: now 1786303708581965 us; error 36 us; skew 500 ppm
Aug 09 19:28:28 dist-test-slave-njxd krb5kdc[5667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303708, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:28.584511 5935 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:28:28.584954 5935 webserver.cc:460] Webserver started at http://127.21.81.194:41037/ using document root <none> and password file <none>
I20260809 19:28:28.585227 5935 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:28.585302 5935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:28.585460 5935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:28:28.586556 5935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "dc44732627e0464c940c34e404b5008b"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "68beaa04316269858fe1ebde27236a2f"
server_key_iv: "d68fccb93408d5a628d4eb9f1b13583d"
server_key_version: "encryptionkey@0"
I20260809 19:28:28.586988 5935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "dc44732627e0464c940c34e404b5008b"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "68beaa04316269858fe1ebde27236a2f"
server_key_iv: "d68fccb93408d5a628d4eb9f1b13583d"
server_key_version: "encryptionkey@0"
I20260809 19:28:28.588284 5935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:28:28.588989 5951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:28.589282 5935 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:28.589375 5935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "dc44732627e0464c940c34e404b5008b"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "68beaa04316269858fe1ebde27236a2f"
server_key_iv: "d68fccb93408d5a628d4eb9f1b13583d"
server_key_version: "encryptionkey@0"
I20260809 19:28:28.589457 5935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:28.610158 5935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:28.612537 5935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:28.612711 5935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:28.612948 5935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:28.613283 5935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:28.613340 5935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:28.613379 5935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:28.613404 5935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:28.619225 5935 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:39347
I20260809 19:28:28.619266 6070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:39347 every 8 connection(s)
I20260809 19:28:28.619583 5935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:28:28.623869 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 5935
I20260809 19:28:28.623982 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:28:28.624209 21831 external_mini_cluster.cc:1428] Setting key 4294802e1b4843afa5cbc1f40d094005
Aug 09 19:28:28 dist-test-slave-njxd krb5kdc[5667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303708, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:28.629686 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.620400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:42107 (local address 127.21.81.254:39717)
0809 19:28:28.620469 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:28.620471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:28.620658 (+ 187us) server_negotiation.cc:407] Connection header received
0809 19:28:28.621161 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:28.621162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:28.621172 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:28.621230 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:28.622234 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.622730 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.623715 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.624014 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.624096 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.626660 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:28.626662 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:28.626662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:28.626671 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:28.628159 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:28.628660 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:28.628661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:28.628661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:28.628712 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:28.628993 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:28.628993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:28.628993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:28.629091 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:28.629142 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:28.629479 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 19:28:28.629540 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:28:28.629966 6073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.620347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:42107)
0809 19:28:28.620525 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:28.620567 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:28.621074 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:28.621280 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:28.621283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:28.621955 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:28.622145 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.622150 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.622812 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.622813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.623606 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.623608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.624103 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.624104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.624211 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.624812 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:28.624820 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:28.626561 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:28.628255 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:28.628262 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:28.628264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:28.628579 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:28.628782 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:28.628782 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:28.628783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:28.628927 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:28.629161 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:28.629168 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:28.629332 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 19:28:28.629464 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 19:28:28.629687 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260809 19:28:28.630477 6071 heartbeater.cc:347] Connected to a master server at 127.21.81.254:39717
I20260809 19:28:28.630587 6071 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:28.630796 6071 heartbeater.cc:511] Master 127.21.81.254:39717 requested a full tablet report, sending...
I20260809 19:28:28.631223 5719 ts_manager.cc:194] Registered new tserver with Master: dc44732627e0464c940c34e404b5008b (127.21.81.194:39347)
I20260809 19:28:28.631608 5719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:42107
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:28:28.653308 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39717
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:28.733753 6077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:28.733942 6077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:28.733963 6077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:28.735122 6077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:28.735165 6077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:28.735191 6077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:28:28.736689 6077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:28.736927 6077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:28.737165 6077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:28.742095 6084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:28.742136 6083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:28.742324 6077 server_base.cc:1034] running on GCE node
W20260809 19:28:28.742105 6086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:28.742666 6077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:28.742911 6077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:28.744078 6077 hybrid_clock.cc:648] HybridClock initialized: now 1786303708744064 us; error 26 us; skew 500 ppm
Aug 09 19:28:28 dist-test-slave-njxd krb5kdc[5667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303708, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:28.746601 6077 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:28:28.747061 6077 webserver.cc:460] Webserver started at http://127.21.81.195:34653/ using document root <none> and password file <none>
I20260809 19:28:28.747313 6077 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:28.747383 6077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:28.747534 6077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:28:28.748519 6077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "62a76f556e7d472b858da8d7bfb75cfa"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "0872068d888ed7cd1c50c8d9eedc12ab"
server_key_iv: "bdc381a68edc73beeafbda3f59ba724f"
server_key_version: "encryptionkey@0"
I20260809 19:28:28.748910 6077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "62a76f556e7d472b858da8d7bfb75cfa"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "0872068d888ed7cd1c50c8d9eedc12ab"
server_key_iv: "bdc381a68edc73beeafbda3f59ba724f"
server_key_version: "encryptionkey@0"
I20260809 19:28:28.750283 6077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:28:28.751001 6094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:28.751178 6077 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:28:28.751289 6077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "62a76f556e7d472b858da8d7bfb75cfa"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "0872068d888ed7cd1c50c8d9eedc12ab"
server_key_iv: "bdc381a68edc73beeafbda3f59ba724f"
server_key_version: "encryptionkey@0"
I20260809 19:28:28.751363 6077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:28.761888 6077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:28.763731 6077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:28.763893 6077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:28.764104 6077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:28.764400 6077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:28.764454 6077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:28.764494 6077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:28.764523 6077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:28.770745 6077 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:39165
I20260809 19:28:28.770793 6216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:39165 every 8 connection(s)
I20260809 19:28:28.771130 6077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:28:28 dist-test-slave-njxd krb5kdc[5667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303708, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:28.778440 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 6077
I20260809 19:28:28.778555 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:28:28.778753 21831 external_mini_cluster.cc:1428] Setting key 22582ca7a2a4fde7367ae2f3c4f63881
I20260809 19:28:28.780592 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.771875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:51899 (local address 127.21.81.254:39717)
0809 19:28:28.771921 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:28.771922 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:28.772131 (+ 209us) server_negotiation.cc:407] Connection header received
0809 19:28:28.772592 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:28.772592 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:28.772602 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:28.772671 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:28.773556 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.774123 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.775110 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.775306 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.775361 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.777487 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:28.777489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:28.777489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:28.777495 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:28.779079 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:28.779574 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:28.779575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:28.779576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:28.779654 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:28.779925 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:28.779925 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:28.779926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:28.780041 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:28.780124 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:28.780419 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 19:28:28.780502 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:28:28.780787 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.771828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:51899)
0809 19:28:28.771994 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:28.772035 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:28.772505 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:28.772724 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:28.772727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:28.773285 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:28.773471 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.773475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.774215 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.774216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.775021 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.775023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.775384 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.775385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.775490 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.775941 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:28.775950 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:28.777366 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:28.779208 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:28.779216 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:28.779218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:28.779496 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:28.779734 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:28.779735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:28.779736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:28.779870 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:28.780125 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:28.780133 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:28.780273 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 19:28:28.780383 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 19:28:28.780546 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260809 19:28:28.781271 6217 heartbeater.cc:347] Connected to a master server at 127.21.81.254:39717
I20260809 19:28:28.781349 6217 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:28.781550 6217 heartbeater.cc:511] Master 127.21.81.254:39717 requested a full tablet report, sending...
I20260809 19:28:28.781957 5719 ts_manager.cc:194] Registered new tserver with Master: 62a76f556e7d472b858da8d7bfb75cfa (127.21.81.195:39165)
I20260809 19:28:28.782348 5719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:51899
I20260809 19:28:28.790628 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:28:28.797360 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.791917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59780 (local address 127.21.81.254:39717)
0809 19:28:28.791957 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:28.791961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:28.792088 (+ 127us) server_negotiation.cc:407] Connection header received
0809 19:28:28.792153 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:28.792154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:28.792162 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:28.792214 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:28.792726 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.793231 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.793989 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.794164 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.794220 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.795273 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:28.795274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:28.795274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:28.795280 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:28.796410 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:28.796725 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:28.796726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:28.796726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:28.796774 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:28.796940 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:28.796941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:28.796941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:28.797072 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:28.797162 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:28.797246 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 19:28:28.797302 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:28:28.798839 5719 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:28:28.800307 5719 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:28:28.807934 6225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.803357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:39347 (local address 127.0.0.1:45284)
0809 19:28:28.803524 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:28.803575 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:28.803649 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:28.804077 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:28.804081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:28.804083 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:28.804148 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.804150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.805481 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.805482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.806783 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.806786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.807619 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.807620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.807698 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.807711 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:28:28.807787 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 19:28:28.807851 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":3,"thread_start_us":60,"threads_started":1}
I20260809 19:28:28.807960 6226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.803555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45284 (local address 127.21.81.194:39347)
0809 19:28:28.803811 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:28.803817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:28.803828 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:28:28.803870 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:28.803873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:28.803889 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:28.803981 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:28.804246 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.805361 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.806877 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.807530 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.807604 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.807661 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:28.807815 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 19:28:28.807868 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":155,"threads_started":1}
I20260809 19:28:28.808341 6227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.804023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:46599 (local address 127.0.0.1:55792)
0809 19:28:28.804238 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:28.804250 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:28.804305 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:28.804836 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:28.804837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:28.804839 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:28.804907 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.804909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.806052 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.806053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.807035 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.807038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.808078 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.808079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.808156 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.808160 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:28:28.808243 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:28:28.808281 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":71,"threads_started":1}
I20260809 19:28:28.808390 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.804142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55792 (local address 127.21.81.193:46599)
0809 19:28:28.804665 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:28.804670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:28.804680 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:28:28.804722 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:28.804724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:28.804738 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:28.804814 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:28.804983 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.805929 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.807146 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.807909 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.807971 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.808027 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:28.808263 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 19:28:28.808306 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":425,"threads_started":1}
I20260809 19:28:28.808610 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.804220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:39165 (local address 127.0.0.1:39868)
0809 19:28:28.804689 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:28.804699 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:28.804751 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:28.805021 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:28.805021 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:28.805023 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:28.805097 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.805099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.806308 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.806309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.807328 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.807331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.808342 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.808343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.808420 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.808425 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:28:28.808475 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 19:28:28.808519 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":3,"thread_start_us":71,"threads_started":1}
I20260809 19:28:28.808837 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.804502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.21.81.195:39165)
0809 19:28:28.804878 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:28.804883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:28.804893 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:28:28.804927 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:28.804929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:28.804944 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:28.805024 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:28.805159 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.806187 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.807425 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.808166 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.808251 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.808309 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:28.808710 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 19:28:28.808756 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":95,"threads_started":1}
I20260809 19:28:28.809085 5853 tablet_service.cc:1467] Processing CreateTablet for tablet 678ac74e77ae43708a27ec101085e406 (DEFAULT_TABLE table=test-table [id=1cb51eac597d4964b5294086de16af72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:28:28.809463 5853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 678ac74e77ae43708a27ec101085e406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:28.809370 6003 tablet_service.cc:1467] Processing CreateTablet for tablet 678ac74e77ae43708a27ec101085e406 (DEFAULT_TABLE table=test-table [id=1cb51eac597d4964b5294086de16af72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:28:28.809722 6003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 678ac74e77ae43708a27ec101085e406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:28.812393 6231 tablet_bootstrap.cc:492] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: Bootstrap starting.
I20260809 19:28:28.812597 6232 tablet_bootstrap.cc:492] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b: Bootstrap starting.
I20260809 19:28:28.813144 6232 tablet_bootstrap.cc:654] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:28.813017 6146 tablet_service.cc:1467] Processing CreateTablet for tablet 678ac74e77ae43708a27ec101085e406 (DEFAULT_TABLE table=test-table [id=1cb51eac597d4964b5294086de16af72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:28:28.813154 6231 tablet_bootstrap.cc:654] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:28.813390 6146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 678ac74e77ae43708a27ec101085e406. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:28.813807 6232 log.cc:824] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:28.813827 6231 log.cc:824] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:28.814693 6232 tablet_bootstrap.cc:492] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b: No bootstrap required, opened a new log
I20260809 19:28:28.814770 6232 ts_tablet_manager.cc:1397] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:28:28.816891 6237 tablet_bootstrap.cc:492] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa: Bootstrap starting.
I20260809 19:28:28.816874 6232 raft_consensus.cc:348] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } }
I20260809 19:28:28.817081 6232 raft_consensus.cc:374] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:28.817138 6232 raft_consensus.cc:729] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc44732627e0464c940c34e404b5008b, State: Initialized, Role: FOLLOWER
I20260809 19:28:28.817256 6232 consensus_queue.cc:260] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } }
I20260809 19:28:28.817619 6071 heartbeater.cc:503] Master 127.21.81.254:39717 was elected leader, sending a full tablet report...
I20260809 19:28:28.817642 6232 ts_tablet_manager.cc:1428] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:28:28.817941 6237 tablet_bootstrap.cc:654] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:28.817994 6231 tablet_bootstrap.cc:492] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: No bootstrap required, opened a new log
I20260809 19:28:28.818070 6231 ts_tablet_manager.cc:1397] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260809 19:28:28.818619 6237 log.cc:824] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:28.819408 6237 tablet_bootstrap.cc:492] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa: No bootstrap required, opened a new log
I20260809 19:28:28.819478 6237 ts_tablet_manager.cc:1397] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:28:28.820031 6231 raft_consensus.cc:348] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } }
I20260809 19:28:28.820211 6231 raft_consensus.cc:374] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:28.820258 6231 raft_consensus.cc:729] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783324c00a734f879546e2d6421d5023, State: Initialized, Role: FOLLOWER
I20260809 19:28:28.820385 6231 consensus_queue.cc:260] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } }
I20260809 19:28:28.820694 6231 ts_tablet_manager.cc:1428] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:28:28.820696 5930 heartbeater.cc:503] Master 127.21.81.254:39717 was elected leader, sending a full tablet report...
I20260809 19:28:28.821022 6237 raft_consensus.cc:348] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } }
I20260809 19:28:28.821226 6237 raft_consensus.cc:374] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:28.821271 6237 raft_consensus.cc:729] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a76f556e7d472b858da8d7bfb75cfa, State: Initialized, Role: FOLLOWER
I20260809 19:28:28.821400 6237 consensus_queue.cc:260] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } }
I20260809 19:28:28.821698 6217 heartbeater.cc:503] Master 127.21.81.254:39717 was elected leader, sending a full tablet report...
I20260809 19:28:28.821703 6237 ts_tablet_manager.cc:1428] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260809 19:28:28.870187 6072 tablet.cc:2369] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:28:28.874943 6240 raft_consensus.cc:482] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:28:28.875056 6240 raft_consensus.cc:504] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } }
I20260809 19:28:28.875382 6240 leader_election.cc:288] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc44732627e0464c940c34e404b5008b (127.21.81.194:39347), 62a76f556e7d472b858da8d7bfb75cfa (127.21.81.195:39165)
I20260809 19:28:28.878640 5932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.875602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:39165 (local address 127.21.81.193:58157)
0809 19:28:28.875687 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:28.875700 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:28.875763 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:28.875990 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:28.875991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:28.875993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:28.876077 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.876079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.876924 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.876925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.877726 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.877728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.878388 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.878389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.878471 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.878476 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:28:28.878530 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 19:28:28.878573 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 19:28:28.878712 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.875673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:58157 (local address 127.21.81.195:39165)
0809 19:28:28.875727 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:28.875730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:28.875749 (+ 19us) server_negotiation.cc:407] Connection header received
0809 19:28:28.875865 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:28.875866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:28.875877 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:28.875947 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:28.876185 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.876839 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.877813 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.878284 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.878339 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.878346 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:28.878577 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 19:28:28.878617 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 19:28:28.878842 6242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.875613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:39347 (local address 127.21.81.193:33021)
0809 19:28:28.875899 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:28.875908 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:28.875966 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:28.876231 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:28.876232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:28.876235 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:28.876306 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.876308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.877044 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.877044 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.877926 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:28.877929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.878563 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.878564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:28.878637 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.878641 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:28:28.878699 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 19:28:28.878739 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":69,"thread_start_us":130,"threads_started":1}
I20260809 19:28:28.878968 6226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.875675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:33021 (local address 127.21.81.194:39347)
0809 19:28:28.875731 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:28.875733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:28.875979 (+ 246us) server_negotiation.cc:407] Connection header received
0809 19:28:28.876023 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:28.876024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:28.876036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:28.876090 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:28.876377 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.876936 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.878015 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.878465 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.878540 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.878548 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:28.878873 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 19:28:28.878911 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:28:28.878998 6169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "678ac74e77ae43708a27ec101085e406" candidate_uuid: "783324c00a734f879546e2d6421d5023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a76f556e7d472b858da8d7bfb75cfa" is_pre_election: true
I20260809 19:28:28.879148 6169 raft_consensus.cc:2454] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 783324c00a734f879546e2d6421d5023 in term 0.
I20260809 19:28:28.879278 6024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "678ac74e77ae43708a27ec101085e406" candidate_uuid: "783324c00a734f879546e2d6421d5023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc44732627e0464c940c34e404b5008b" is_pre_election: true
I20260809 19:28:28.879412 6024 raft_consensus.cc:2454] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 783324c00a734f879546e2d6421d5023 in term 0.
I20260809 19:28:28.879392 5806 leader_election.cc:302] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a76f556e7d472b858da8d7bfb75cfa, 783324c00a734f879546e2d6421d5023; no voters:
I20260809 19:28:28.879519 6240 raft_consensus.cc:2781] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:28:28.879604 6240 raft_consensus.cc:482] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:28:28.879632 6240 raft_consensus.cc:3037] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:28.880409 6240 raft_consensus.cc:504] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } }
I20260809 19:28:28.880555 6240 leader_election.cc:288] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 1 election: Requested vote from peers dc44732627e0464c940c34e404b5008b (127.21.81.194:39347), 62a76f556e7d472b858da8d7bfb75cfa (127.21.81.195:39165)
I20260809 19:28:28.880702 6024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "678ac74e77ae43708a27ec101085e406" candidate_uuid: "783324c00a734f879546e2d6421d5023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc44732627e0464c940c34e404b5008b"
I20260809 19:28:28.880775 6024 raft_consensus.cc:3037] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:28.881487 6024 raft_consensus.cc:2454] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 783324c00a734f879546e2d6421d5023 in term 1.
I20260809 19:28:28.881672 5803 leader_election.cc:302] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783324c00a734f879546e2d6421d5023, dc44732627e0464c940c34e404b5008b; no voters:
I20260809 19:28:28.881675 6169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "678ac74e77ae43708a27ec101085e406" candidate_uuid: "783324c00a734f879546e2d6421d5023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a76f556e7d472b858da8d7bfb75cfa"
I20260809 19:28:28.881762 6240 raft_consensus.cc:2781] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:28:28.881785 6169 raft_consensus.cc:3037] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:28.881913 6240 raft_consensus.cc:686] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [term 1 LEADER]: Becoming Leader. State: Replica: 783324c00a734f879546e2d6421d5023, State: Running, Role: LEADER
I20260809 19:28:28.882045 6240 consensus_queue.cc:237] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } }
I20260809 19:28:28.882687 6169 raft_consensus.cc:2454] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 783324c00a734f879546e2d6421d5023 in term 1.
I20260809 19:28:28.882932 5719 catalog_manager.cc:5495] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 reported cstate change: term changed from 0 to 1, leader changed from <none> to 783324c00a734f879546e2d6421d5023 (127.21.81.193). New cstate: current_term: 1 leader_uuid: "783324c00a734f879546e2d6421d5023" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } health_report { overall_health: UNKNOWN } } }
I20260809 19:28:28.899477 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.896843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55806 (local address 127.21.81.193:46599)
0809 19:28:28.896893 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:28.896895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:28.896961 (+ 66us) server_negotiation.cc:407] Connection header received
0809 19:28:28.897046 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:28.897052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:28.897083 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:28.897166 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:28:28.897312 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.897854 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.898632 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:28.898832 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:28.898913 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:28.899115 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:28:28.899216 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:28:28.899315 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:28.899339 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 19:28:28.899360 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:28:28.901463 6024 raft_consensus.cc:1261] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b [term 1 FOLLOWER]: Refusing update from remote peer 783324c00a734f879546e2d6421d5023: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:28:28.901461 6169 raft_consensus.cc:1261] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa [term 1 FOLLOWER]: Refusing update from remote peer 783324c00a734f879546e2d6421d5023: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:28:28.901894 6243 consensus_queue.cc:1035] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:28:28.902042 6240 consensus_queue.cc:1035] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:28:28.904578 6246 mvcc.cc:205] Tried to move back new op lower bound from 7316699991657377792 to 7316699991581421568. Current Snapshot: MvccSnapshot[applied={T|T < 7316699991657377792}]
I20260809 19:28:28.904824 6248 mvcc.cc:205] Tried to move back new op lower bound from 7316699991657377792 to 7316699991581421568. Current Snapshot: MvccSnapshot[applied={T|T < 7316699991657377792}]
I20260809 19:28:28.904942 6247 mvcc.cc:205] Tried to move back new op lower bound from 7316699991657377792 to 7316699991581421568. Current Snapshot: MvccSnapshot[applied={T|T < 7316699991657377792}]
I20260809 19:28:28.907442 5719 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:28:28.907555 5719 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:28:28.908358 5719 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Sending DeleteTablet for 3 replicas of tablet 678ac74e77ae43708a27ec101085e406
I20260809 19:28:28.908691 6003 tablet_service.cc:1514] Processing DeleteTablet for tablet 678ac74e77ae43708a27ec101085e406 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:28 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:45284
I20260809 19:28:28.908777 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 5785
I20260809 19:28:28.908766 6146 tablet_service.cc:1514] Processing DeleteTablet for tablet 678ac74e77ae43708a27ec101085e406 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:28 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:39868
I20260809 19:28:28.910502 6261 tablet_replica.cc:330] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b: stopping tablet replica
I20260809 19:28:28.910584 6261 raft_consensus.cc:2229] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:28:28.910656 6261 raft_consensus.cc:2258] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:28:28.911190 6263 tablet_replica.cc:330] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa: stopping tablet replica
I20260809 19:28:28.911218 6261 ts_tablet_manager.cc:1905] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:28:28.911295 6263 raft_consensus.cc:2229] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:28:28.911392 6263 raft_consensus.cc:2258] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:28:28.911863 6263 ts_tablet_manager.cc:1905] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:28:28.912892 6261 ts_tablet_manager.cc:1918] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:28:28.912945 6261 log.cc:1196] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/678ac74e77ae43708a27ec101085e406
I20260809 19:28:28.913151 6261 ts_tablet_manager.cc:1939] T 678ac74e77ae43708a27ec101085e406 P dc44732627e0464c940c34e404b5008b: Deleting consensus metadata
I20260809 19:28:28.913607 5702 catalog_manager.cc:4841] TS dc44732627e0464c940c34e404b5008b (127.21.81.194:39347): tablet 678ac74e77ae43708a27ec101085e406 (table test-table [id=1cb51eac597d4964b5294086de16af72]) successfully deleted
I20260809 19:28:28.913853 6263 ts_tablet_manager.cc:1918] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:28:28.913931 6263 log.cc:1196] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/678ac74e77ae43708a27ec101085e406
I20260809 19:28:28.914211 6263 ts_tablet_manager.cc:1939] T 678ac74e77ae43708a27ec101085e406 P 62a76f556e7d472b858da8d7bfb75cfa: Deleting consensus metadata
I20260809 19:28:28.914662 5704 catalog_manager.cc:4841] TS 62a76f556e7d472b858da8d7bfb75cfa (127.21.81.195:39165): tablet 678ac74e77ae43708a27ec101085e406 (table test-table [id=1cb51eac597d4964b5294086de16af72]) successfully deleted
W20260809 19:28:28.918011 5704 connection.cc:535] client connection to 127.21.81.193:46599 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 19:28:28.918116 5704 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 19:28:28.918381 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 5935
I20260809 19:28:28.920413 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:28.918511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:46599 (local address 127.0.0.1:55814)
0809 19:28:28.920329 (+ 1818us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:28.920373 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:46599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1797}
W20260809 19:28:28.920594 5704 catalog_manager.cc:4584] TS 783324c00a734f879546e2d6421d5023 (127.21.81.193:46599): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 678ac74e77ae43708a27ec101085e406: Network error: Client connection negotiation failed: client connection to 127.21.81.193:46599: connect: Connection refused (error 111)
I20260809 19:28:28.925786 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 6077
I20260809 19:28:28.932221 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 5683
I20260809 19:28:48.938931 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39717
--webserver_interface=127.21.81.254
--webserver_port=46309
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:39717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:28:49.022727 6265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:49.022936 6265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:49.022959 6265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:49.024133 6265 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 19:28:49.024175 6265 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 19:28:49.024189 6265 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 19:28:49.024204 6265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:28:49.024216 6265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:49.024271 6265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:49.024287 6265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:28:49.024310 6265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:28:49.025960 6265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:39717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=46309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:49.026219 6265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:49.026484 6265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:49.035822 6275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:49.035780 6271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:49.035780 6272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:49.035998 6265 server_base.cc:1034] running on GCE node
I20260809 19:28:49.036439 6265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:49.036715 6265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:49.037851 6265 hybrid_clock.cc:648] HybridClock initialized: now 1786303729037840 us; error 26 us; skew 500 ppm
Aug 09 19:28:49 dist-test-slave-njxd krb5kdc[5667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:49.040541 6265 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:28:49.041005 6265 webserver.cc:460] Webserver started at http://127.21.81.254:46309/ using document root <none> and password file <none>
I20260809 19:28:49.041256 6265 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:49.041330 6265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:49.042771 6265 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:49.043418 6283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:49.043588 6265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:28:49.043674 6265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "b13d62f274a24b66923d5daaf5ba7b9a"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "102d37588b3fcc67bb4c84b0f3887f1f"
server_key_iv: "9e7e279984ed654778d2907f121f7419"
server_key_version: "encryptionkey@0"
I20260809 19:28:49.044061 6265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:49.065855 6265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:49.067781 6265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:49.067890 6265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:49.073380 6265 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:39717
I20260809 19:28:49.073428 6337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:39717 every 8 connection(s)
I20260809 19:28:49.073976 6265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:28:49.075412 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 6265
I20260809 19:28:49.075737 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:46599
--local_ip_for_outbound_sockets=127.21.81.193
--tserver_master_addrs=127.21.81.254:39717
--webserver_port=40363
--webserver_interface=127.21.81.193
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:28:49.076040 6338 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:28:49.077178 6338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Bootstrap starting.
I20260809 19:28:49.086658 6338 log.cc:824] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:49.088634 6338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:28:49.088964 6338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Bootstrap complete.
I20260809 19:28:49.091353 6338 raft_consensus.cc:348] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } }
I20260809 19:28:49.091599 6338 raft_consensus.cc:729] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13d62f274a24b66923d5daaf5ba7b9a, State: Initialized, Role: FOLLOWER
I20260809 19:28:49.091790 6338 consensus_queue.cc:260] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } }
I20260809 19:28:49.091849 6338 raft_consensus.cc:388] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:28:49.091919 6338 raft_consensus.cc:482] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:28:49.091969 6338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:28:49.092950 6338 raft_consensus.cc:504] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } }
I20260809 19:28:49.093031 6338 leader_election.cc:302] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b13d62f274a24b66923d5daaf5ba7b9a; no voters:
I20260809 19:28:49.093253 6338 leader_election.cc:288] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 19:28:49.093295 6343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:28:49.093433 6343 raft_consensus.cc:686] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 2 LEADER]: Becoming Leader. State: Replica: b13d62f274a24b66923d5daaf5ba7b9a, State: Running, Role: LEADER
I20260809 19:28:49.093515 6338 sys_catalog.cc:564] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:28:49.093525 6343 consensus_queue.cc:237] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } }
I20260809 19:28:49.093959 6344 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b13d62f274a24b66923d5daaf5ba7b9a. Latest consensus state: current_term: 2 leader_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } } }
I20260809 19:28:49.094056 6344 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:49.094007 6345 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } } }
I20260809 19:28:49.094308 6345 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [sys.catalog]: This master's current role is: LEADER
I20260809 19:28:49.095320 6360 catalog_manager.cc:1259] Loaded cluster ID: 29735a69af56418688e2234ced76f6d5
I20260809 19:28:49.095358 6360 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: loading cluster ID for follower catalog manager: success
I20260809 19:28:49.096091 6360 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: acquiring CA information for follower catalog manager: success
I20260809 19:28:49.096411 6360 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:28:49.096513 6351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:28:49.096904 6351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:28:49.096988 6351 catalog_manager.cc:1259] Loaded cluster ID: 29735a69af56418688e2234ced76f6d5
I20260809 19:28:49.097020 6351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:28:49.097170 6351 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:28:49.097256 6351 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Loaded TSK: 0
I20260809 19:28:49.108273 6351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Generated new TSK 1
I20260809 19:28:49.108371 6351 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 19:28:49.162626 6340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:49.162853 6340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:49.162894 6340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:49.164083 6340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:49.164151 6340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:49.164197 6340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:28:49.165787 6340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:46599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=40363
--tserver_master_addrs=127.21.81.254:39717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:49.166039 6340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:49.166270 6340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:49.172335 6368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:49.172365 6367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:49.172336 6370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:49.173012 6340 server_base.cc:1034] running on GCE node
I20260809 19:28:49.173220 6340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:49.173460 6340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:49.174649 6340 hybrid_clock.cc:648] HybridClock initialized: now 1786303729174636 us; error 59 us; skew 500 ppm
Aug 09 19:28:49 dist-test-slave-njxd krb5kdc[5667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:49.177250 6340 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:28:49.177811 6340 webserver.cc:460] Webserver started at http://127.21.81.193:40363/ using document root <none> and password file <none>
I20260809 19:28:49.178071 6340 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:49.178128 6340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:49.179497 6340 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:49.180256 6377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:49.180449 6340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:49.180572 6340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "783324c00a734f879546e2d6421d5023"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "33e9ad57d392ac2473afab2a1dda6e2b"
server_key_iv: "1b015aea426213b1c97b5db38c0658f4"
server_key_version: "encryptionkey@0"
I20260809 19:28:49.180929 6340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:49.208550 6340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:49.211112 6340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:49.211299 6340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:49.211678 6340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:49.212220 6385 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:28:49.214255 6340 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:28:49.214330 6340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:28:49.214378 6340 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:28:49.215077 6340 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:28:49.215134 6340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 19:28:49.215226 6385 tablet_bootstrap.cc:492] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: Bootstrap starting.
I20260809 19:28:49.222841 6340 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:46599
I20260809 19:28:49.223259 6340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:28:49.226543 6385 log.cc:824] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:49.226958 6497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:46599 every 8 connection(s)
I20260809 19:28:49.229429 6385 tablet_bootstrap.cc:492] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:28:49.229794 6385 tablet_bootstrap.cc:492] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: Bootstrap complete.
I20260809 19:28:49.231319 6385 ts_tablet_manager.cc:1397] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260809 19:28:49.232237 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 6340
I20260809 19:28:49.232549 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:39347
--local_ip_for_outbound_sockets=127.21.81.194
--tserver_master_addrs=127.21.81.254:39717
--webserver_port=41037
--webserver_interface=127.21.81.194
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:28:49.232759 6385 raft_consensus.cc:348] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } }
I20260809 19:28:49.233063 6385 raft_consensus.cc:729] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783324c00a734f879546e2d6421d5023, State: Initialized, Role: FOLLOWER
I20260809 19:28:49.233225 6385 consensus_queue.cc:260] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } }
I20260809 19:28:49.233494 6385 ts_tablet_manager.cc:1428] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
Aug 09 19:28:49 dist-test-slave-njxd krb5kdc[5667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:49.238718 6500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.224027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:37003)
0809 19:28:49.227195 (+ 3168us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:49.227257 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:49.227819 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:49.228287 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:49.228291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:49.228584 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:49.228788 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.228794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.230069 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.230070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.230903 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.230905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.231548 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.231549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.231688 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.232566 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:49.232577 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:49.235057 (+ 2480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:49.236977 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:49.236984 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:49.236986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:49.237284 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:49.237545 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:49.237546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:49.237547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:49.237698 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:49.238088 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:49.238095 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:49.238238 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 19:28:49.238337 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 19:28:49.238520 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3119,"thread_start_us":110,"threads_started":1}
I20260809 19:28:49.238725 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.224121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:37003 (local address 127.21.81.254:39717)
0809 19:28:49.227297 (+ 3176us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:49.227308 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:49.227379 (+ 71us) server_negotiation.cc:407] Connection header received
0809 19:28:49.228059 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:49.228064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:49.228109 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:49.228257 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:49.228918 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.229958 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.230999 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.231443 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.231555 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.235180 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:49.235184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:49.235184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:49.235194 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:49.236889 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:49.237367 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:49.237370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:49.237370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:49.237454 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:49.237772 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:49.237773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:49.237773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:49.238001 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:49.238095 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:49.238409 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 19:28:49.238536 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3117,"thread_start_us":81,"threads_started":1}
I20260809 19:28:49.239377 6498 heartbeater.cc:347] Connected to a master server at 127.21.81.254:39717
I20260809 19:28:49.239473 6498 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:49.239674 6498 heartbeater.cc:511] Master 127.21.81.254:39717 requested a full tablet report, sending...
I20260809 19:28:49.240394 6301 ts_manager.cc:194] Registered new tserver with Master: 783324c00a734f879546e2d6421d5023 (127.21.81.193:46599)
I20260809 19:28:49.241647 6301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:37003
I20260809 19:28:49.242578 6498 heartbeater.cc:503] Master 127.21.81.254:39717 was elected leader, sending a full tablet report...
Aug 09 19:28:49 dist-test-slave-njxd krb5kdc[5667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for kudu/127.21.81.193@KRBTEST.COM
I20260809 19:28:49.249450 6506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.241253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60986 (local address 127.21.81.193:46599)
0809 19:28:49.241400 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:49.241404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:49.241662 (+ 258us) server_negotiation.cc:407] Connection header received
0809 19:28:49.241699 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:49.241778 (+ 79us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:49.241798 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:49.241864 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:49.242489 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.243376 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.244126 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.244428 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.244508 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.246512 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:49.246520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:49.246520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:49.246527 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:49.248183 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:49.248577 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:49.248580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:49.248580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:49.248644 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:49.248850 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:49.248851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:49.248852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:49.248987 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:49.249074 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:49.249276 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 19:28:49.249365 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
I20260809 19:28:49.249450 6505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.241003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:46599 (local address 127.0.0.1:60986)
0809 19:28:49.241460 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:49.241493 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:49.241561 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:49.242007 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:49.242010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:49.242332 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:49.242387 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.242390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.243461 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.243463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.244027 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.244028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.244507 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.244508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.244591 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.245103 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:49.245107 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:49.246427 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:49.248278 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:49.248286 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:49.248287 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:49.248482 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:49.248721 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:49.248722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:49.248722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:49.248787 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:49.249066 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:49.249073 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:49.249128 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 19:28:49.249226 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 19:28:49.249352 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":89,"threads_started":1}
I20260809 19:28:49.249950 6431 tablet_service.cc:1514] Processing DeleteTablet for tablet 678ac74e77ae43708a27ec101085e406 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:28 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:60986
I20260809 19:28:49.250458 6507 tablet_replica.cc:330] stopping tablet replica
I20260809 19:28:49.250576 6507 raft_consensus.cc:2229] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:28:49.250646 6507 raft_consensus.cc:2258] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:28:49.250973 6507 ts_tablet_manager.cc:1905] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:28:49.252890 6507 ts_tablet_manager.cc:1918] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:28:49.252954 6507 log.cc:1196] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal/wals/678ac74e77ae43708a27ec101085e406
I20260809 19:28:49.253206 6507 ts_tablet_manager.cc:1939] T 678ac74e77ae43708a27ec101085e406 P 783324c00a734f879546e2d6421d5023: Deleting consensus metadata
I20260809 19:28:49.253744 6287 catalog_manager.cc:4841] TS 783324c00a734f879546e2d6421d5023 (127.21.81.193:46599): tablet 678ac74e77ae43708a27ec101085e406 (table test-table [id=1cb51eac597d4964b5294086de16af72]) successfully deleted
W20260809 19:28:49.321116 6504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:49.321368 6504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:49.321408 6504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:49.322680 6504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:49.322752 6504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:49.322798 6504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:28:49.324263 6504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:39347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=41037
--tserver_master_addrs=127.21.81.254:39717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:49.324540 6504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:49.324783 6504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:49.329849 6515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:49.329895 6512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:49.329860 6513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:49.330176 6504 server_base.cc:1034] running on GCE node
I20260809 19:28:49.330415 6504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:49.330658 6504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:49.331840 6504 hybrid_clock.cc:648] HybridClock initialized: now 1786303729331828 us; error 29 us; skew 500 ppm
Aug 09 19:28:49 dist-test-slave-njxd krb5kdc[5667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:49.334343 6504 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:28:49.334743 6504 webserver.cc:460] Webserver started at http://127.21.81.194:41037/ using document root <none> and password file <none>
I20260809 19:28:49.334923 6504 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:49.334971 6504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:49.336256 6504 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:28:49.336812 6523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:49.336985 6504 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:28:49.337067 6504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "dc44732627e0464c940c34e404b5008b"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "68beaa04316269858fe1ebde27236a2f"
server_key_iv: "d68fccb93408d5a628d4eb9f1b13583d"
server_key_version: "encryptionkey@0"
I20260809 19:28:49.337385 6504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:49.346882 6504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:49.349171 6504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:49.349289 6504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:49.349539 6504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:49.349920 6504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:49.349956 6504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:49.349977 6504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:49.350032 6504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:49.355937 6504 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:39347
I20260809 19:28:49.356014 6643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:39347 every 8 connection(s)
I20260809 19:28:49.356359 6504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:28:49.361291 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 6504
I20260809 19:28:49.361611 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:39165
--local_ip_for_outbound_sockets=127.21.81.195
--tserver_master_addrs=127.21.81.254:39717
--webserver_port=34653
--webserver_interface=127.21.81.195
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:28:49 dist-test-slave-njxd krb5kdc[5667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:49.365835 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.357110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:41039 (local address 127.21.81.254:39717)
0809 19:28:49.357178 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:49.357181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:49.357446 (+ 265us) server_negotiation.cc:407] Connection header received
0809 19:28:49.357978 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:49.357979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:49.357993 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:49.358046 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:49.358943 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.359422 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.360337 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.360525 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.360574 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.362844 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:49.362846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:49.362846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:49.362856 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:49.364349 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:49.364852 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:49.364853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:49.364853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:49.364904 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:49.365178 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:49.365179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:49.365179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:49.365280 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:49.365331 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:49.365692 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 19:28:49.365771 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 19:28:49.366101 6646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.357107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:41039)
0809 19:28:49.357323 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:49.357361 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:49.357891 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:49.358096 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:49.358100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:49.358662 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:49.358855 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.358860 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.359500 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.359501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.360267 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.360268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.360598 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.360599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.360691 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.361242 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:49.361250 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:49.362748 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:49.364452 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:49.364459 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:49.364461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:49.364774 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:49.364969 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:49.364970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:49.364970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:49.365117 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:49.365344 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:49.365351 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:49.365516 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 19:28:49.365667 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 19:28:49.365866 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":110,"threads_started":1}
I20260809 19:28:49.366686 6644 heartbeater.cc:347] Connected to a master server at 127.21.81.254:39717
I20260809 19:28:49.366798 6644 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:49.367017 6644 heartbeater.cc:511] Master 127.21.81.254:39717 requested a full tablet report, sending...
I20260809 19:28:49.367413 6301 ts_manager.cc:194] Registered new tserver with Master: dc44732627e0464c940c34e404b5008b (127.21.81.194:39347)
I20260809 19:28:49.367985 6301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:41039
W20260809 19:28:49.447024 6647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:28:49.447218 6647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:28:49.447242 6647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:28:49.448421 6647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:28:49.448464 6647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:28:49.448490 6647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:28:49.449905 6647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:39165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=34653
--tserver_master_addrs=127.21.81.254:39717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:28:49.450088 6647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:28:49.450256 6647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:28:49.455112 6652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:49.455101 6655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:28:49.455098 6653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:28:49.455351 6647 server_base.cc:1034] running on GCE node
I20260809 19:28:49.455575 6647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:28:49.455817 6647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:28:49.456962 6647 hybrid_clock.cc:648] HybridClock initialized: now 1786303729456950 us; error 28 us; skew 500 ppm
Aug 09 19:28:49 dist-test-slave-njxd krb5kdc[5667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:28:49.459491 6647 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:28:49.459944 6647 webserver.cc:460] Webserver started at http://127.21.81.195:34653/ using document root <none> and password file <none>
I20260809 19:28:49.460183 6647 fs_manager.cc:359] Metadata directory not provided
I20260809 19:28:49.460271 6647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:28:49.461546 6647 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:28:49.462193 6663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:49.463603 6647 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260809 19:28:49.463709 6647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "62a76f556e7d472b858da8d7bfb75cfa"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "0872068d888ed7cd1c50c8d9eedc12ab"
server_key_iv: "bdc381a68edc73beeafbda3f59ba724f"
server_key_version: "encryptionkey@0"
I20260809 19:28:49.464039 6647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:28:49.478118 6647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:28:49.480511 6647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:28:49.480675 6647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:28:49.480917 6647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:28:49.481235 6647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:28:49.481289 6647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:49.481331 6647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:28:49.481359 6647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:49.488045 6647 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:39165
I20260809 19:28:49.488092 6787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:39165 every 8 connection(s)
I20260809 19:28:49.488456 6647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:28:49 dist-test-slave-njxd krb5kdc[5667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:28:49.497440 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.489151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:55507 (local address 127.21.81.254:39717)
0809 19:28:49.489197 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:49.489199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:49.489425 (+ 226us) server_negotiation.cc:407] Connection header received
0809 19:28:49.489996 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:49.489997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:49.490006 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:49.490069 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:49.491004 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.491600 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.492566 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.492754 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.492803 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.495085 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:49.495086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:49.495086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:49.495093 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:49.496290 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:49.496651 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:49.496652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:49.496652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:49.496709 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:49.496921 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:49.496922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:49.496922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:49.497023 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:49.497072 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:49.497309 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 19:28:49.497373 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:28:49.497632 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.489131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:55507)
0809 19:28:49.489296 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:49.489339 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:49.489907 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:49.490143 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:49.490147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:49.490733 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:28:49.490923 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.490928 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.491678 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.491679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.492466 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.492467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.492823 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.492824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.493002 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.493487 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:28:49.493494 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:28:49.494995 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:28:49.496359 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:49.496366 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:49.496367 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:49.496582 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:49.496768 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:28:49.496768 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:28:49.496768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:28:49.496866 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:28:49.497074 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:28:49.497079 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:28:49.497192 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 19:28:49.497280 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 19:28:49.497402 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260809 19:28:49.498201 6788 heartbeater.cc:347] Connected to a master server at 127.21.81.254:39717
I20260809 19:28:49.498296 6788 heartbeater.cc:464] Registering TS with master...
I20260809 19:28:49.498375 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 6647
I20260809 19:28:49.498524 6788 heartbeater.cc:511] Master 127.21.81.254:39717 requested a full tablet report, sending...
I20260809 19:28:49.498981 6301 ts_manager.cc:194] Registered new tserver with Master: 62a76f556e7d472b858da8d7bfb75cfa (127.21.81.195:39165)
I20260809 19:28:49.499456 6301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:55507
I20260809 19:28:49.504792 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.498836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40172 (local address 127.21.81.254:39717)
0809 19:28:49.498874 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:49.498877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:49.499105 (+ 228us) server_negotiation.cc:407] Connection header received
0809 19:28:49.499186 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:49.499187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:49.499196 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:49.499247 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:49.499927 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.500382 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.501145 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.501336 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.501412 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.502609 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:49.502611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:49.502611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:49.502618 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:49.503642 (+ 1024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:49.503994 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:49.503995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:49.503995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:49.504052 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:49.504252 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:49.504253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:49.504253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:49.504359 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:49.504568 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:49.504609 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 19:28:49.504684 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:28:49.505513 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:28:49.507880 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.505905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40176 (local address 127.21.81.254:39717)
0809 19:28:49.505969 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:49.505975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:49.506007 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:28:49.506064 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:49.506065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:49.506092 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:49.506145 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:28:49.506222 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.506627 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.507363 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.507527 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.507574 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.507704 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:28:49.507741 (+ 37us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 19:28:49.507840 (+ 99us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40176: token expired
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 19:28:49.507953 6501 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40176: token expired
W20260809 19:28:49.508117 6221 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.21.81.254:39717)
I20260809 19:28:49.513499 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.508377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40178 (local address 127.21.81.254:39717)
0809 19:28:49.508425 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:49.508427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:49.508436 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:28:49.508462 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:49.508463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:49.508469 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:49.508509 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:28:49.508929 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.509366 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.510080 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.510244 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.510295 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.511375 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:28:49.511376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:28:49.511376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:28:49.511381 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:28:49.512622 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:49.512920 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:49.512921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:49.512921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:49.512968 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:28:49.513126 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:28:49.513127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:28:49.513127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:28:49.513216 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:28:49.513286 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:49.513365 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 19:28:49.513419 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:28:49.514622 6301 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:28:49.516240 6301 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:28:49.518450 6431 tablet_service.cc:1467] Processing CreateTablet for tablet 491364c292f7440f8f2670a4fda1895d (DEFAULT_TABLE table=test-table [id=4e5411353d95456ca570aaf7ffa63a88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:28:49.518676 6431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 491364c292f7440f8f2670a4fda1895d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:49.520238 6385 tablet_bootstrap.cc:492] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023: Bootstrap starting.
I20260809 19:28:49.520540 6385 tablet_bootstrap.cc:654] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:49.522126 6505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.518004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:39347 (local address 127.0.0.1:44168)
0809 19:28:49.518050 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:49.518061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:49.518097 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:49.518522 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:49.518523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:49.518525 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:49.518571 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.518573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.519908 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.519908 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.521038 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.521040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.521814 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.521815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.521883 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.521905 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 19:28:49.521954 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:28:49.522005 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":29}
I20260809 19:28:49.522140 6794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.518111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44168 (local address 127.21.81.194:39347)
0809 19:28:49.518330 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:49.518336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:49.518346 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:28:49.518387 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:49.518389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:49.518404 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:49.518649 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:49.518769 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.519815 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.521131 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.521735 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.521792 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.521866 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:49.521987 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 19:28:49.522054 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":122,"threads_started":1}
I20260809 19:28:49.522241 6793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.518046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:39165 (local address 127.0.0.1:56596)
0809 19:28:49.518331 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:49.518342 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:49.518394 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:49.518715 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:49.518716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:49.518718 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:49.518789 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.518791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.519864 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.519865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.521123 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.521125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.521843 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.521844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.521909 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.521913 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:28:49.521968 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 19:28:49.522033 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":27,"thread_start_us":56,"threads_started":1}
I20260809 19:28:49.522274 6385 tablet_bootstrap.cc:492] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023: No bootstrap required, opened a new log
I20260809 19:28:49.522327 6385 ts_tablet_manager.cc:1397] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:28:49.522495 6385 raft_consensus.cc:348] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } }
I20260809 19:28:49.522570 6385 raft_consensus.cc:374] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:49.522614 6385 raft_consensus.cc:729] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783324c00a734f879546e2d6421d5023, State: Initialized, Role: FOLLOWER
I20260809 19:28:49.522667 6385 consensus_queue.cc:260] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } }
I20260809 19:28:49.522773 6385 ts_tablet_manager.cc:1428] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:28:49.522725 6796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.518193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56596 (local address 127.21.81.195:39165)
0809 19:28:49.518383 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:49.518389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:49.518400 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:28:49.518466 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:49.518468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:49.518481 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:49.518629 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:49.518892 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.519760 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.521194 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.521745 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.521830 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.521882 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:49.522169 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 19:28:49.522629 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":81,"threads_started":1}
I20260809 19:28:49.523613 6575 tablet_service.cc:1467] Processing CreateTablet for tablet 491364c292f7440f8f2670a4fda1895d (DEFAULT_TABLE table=test-table [id=4e5411353d95456ca570aaf7ffa63a88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:28:49.523576 6716 tablet_service.cc:1467] Processing CreateTablet for tablet 491364c292f7440f8f2670a4fda1895d (DEFAULT_TABLE table=test-table [id=4e5411353d95456ca570aaf7ffa63a88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:28:49.523928 6575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 491364c292f7440f8f2670a4fda1895d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:49.523928 6716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 491364c292f7440f8f2670a4fda1895d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:28:49.527486 6798 tablet_bootstrap.cc:492] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b: Bootstrap starting.
I20260809 19:28:49.528096 6798 tablet_bootstrap.cc:654] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:49.528520 6798 log.cc:824] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:49.528775 6799 tablet_bootstrap.cc:492] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa: Bootstrap starting.
I20260809 19:28:49.529160 6798 tablet_bootstrap.cc:492] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b: No bootstrap required, opened a new log
I20260809 19:28:49.529214 6798 ts_tablet_manager.cc:1397] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:28:49.529343 6799 tablet_bootstrap.cc:654] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa: Neither blocks nor log segments found. Creating new log.
I20260809 19:28:49.529966 6799 log.cc:824] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa: Log is configured to *not* fsync() on all Append() calls
I20260809 19:28:49.530695 6799 tablet_bootstrap.cc:492] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa: No bootstrap required, opened a new log
I20260809 19:28:49.530748 6799 ts_tablet_manager.cc:1397] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:28:49.531149 6798 raft_consensus.cc:348] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } }
I20260809 19:28:49.531304 6798 raft_consensus.cc:374] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:49.531363 6798 raft_consensus.cc:729] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc44732627e0464c940c34e404b5008b, State: Initialized, Role: FOLLOWER
I20260809 19:28:49.531471 6798 consensus_queue.cc:260] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } }
I20260809 19:28:49.531770 6644 heartbeater.cc:503] Master 127.21.81.254:39717 was elected leader, sending a full tablet report...
I20260809 19:28:49.531785 6798 ts_tablet_manager.cc:1428] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:28:49.532343 6799 raft_consensus.cc:348] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } }
I20260809 19:28:49.532524 6799 raft_consensus.cc:374] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:28:49.532573 6799 raft_consensus.cc:729] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a76f556e7d472b858da8d7bfb75cfa, State: Initialized, Role: FOLLOWER
I20260809 19:28:49.532684 6799 consensus_queue.cc:260] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } }
I20260809 19:28:49.532939 6799 ts_tablet_manager.cc:1428] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:28:49.532979 6788 heartbeater.cc:503] Master 127.21.81.254:39717 was elected leader, sending a full tablet report...
I20260809 19:28:49.550974 6803 raft_consensus.cc:482] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:28:49.551064 6803 raft_consensus.cc:504] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } }
I20260809 19:28:49.551436 6803 leader_election.cc:288] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc44732627e0464c940c34e404b5008b (127.21.81.194:39347), 783324c00a734f879546e2d6421d5023 (127.21.81.193:46599)
I20260809 19:28:49.555409 6794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.551748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:50659 (local address 127.21.81.194:39347)
0809 19:28:49.551809 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:49.551811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:49.551960 (+ 149us) server_negotiation.cc:407] Connection header received
0809 19:28:49.552008 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:49.552009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:49.552015 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:49.552053 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:49.552196 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.552915 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.554444 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.555040 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.555125 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.555137 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:49.555301 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 19:28:49.555341 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:28:49.555409 6506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.551690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:48333 (local address 127.21.81.193:46599)
0809 19:28:49.551781 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:49.551783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:49.551795 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:28:49.551825 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:49.551826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:49.551862 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:49.551921 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:28:49.552129 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.552916 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.554445 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.555080 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.555155 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.555231 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:49.555304 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 19:28:49.555346 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:28:49.555410 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.551630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:46599 (local address 127.21.81.195:48333)
0809 19:28:49.551681 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:49.551692 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:49.551738 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:49.551946 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:49.551948 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:49.551949 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:49.552025 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.552026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.552997 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.552998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.554335 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.554338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.555160 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.555161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.555238 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.555241 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:28:49.555287 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 19:28:49.555326 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":67}
I20260809 19:28:49.555421 6804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.551684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:39347 (local address 127.21.81.195:50659)
0809 19:28:49.551875 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:49.551882 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 19:28:49.551946 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:28:49.552070 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:28:49.552071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:28:49.552072 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:28:49.552134 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.552135 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.552997 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.552998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.554348 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:28:49.554350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.555139 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.555140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:28:49.555214 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.555219 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:28:49.555281 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:28:49.555326 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":52,"thread_start_us":117,"threads_started":1}
I20260809 19:28:49.555770 6452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "491364c292f7440f8f2670a4fda1895d" candidate_uuid: "62a76f556e7d472b858da8d7bfb75cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "783324c00a734f879546e2d6421d5023" is_pre_election: true
I20260809 19:28:49.555825 6596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "491364c292f7440f8f2670a4fda1895d" candidate_uuid: "62a76f556e7d472b858da8d7bfb75cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc44732627e0464c940c34e404b5008b" is_pre_election: true
I20260809 19:28:49.555934 6596 raft_consensus.cc:2454] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a76f556e7d472b858da8d7bfb75cfa in term 0.
I20260809 19:28:49.555925 6452 raft_consensus.cc:2454] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a76f556e7d472b858da8d7bfb75cfa in term 0.
I20260809 19:28:49.556182 6665 leader_election.cc:302] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a76f556e7d472b858da8d7bfb75cfa, dc44732627e0464c940c34e404b5008b; no voters:
I20260809 19:28:49.556318 6803 raft_consensus.cc:2781] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:28:49.556373 6803 raft_consensus.cc:482] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:28:49.556396 6803 raft_consensus.cc:3037] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:49.557075 6803 raft_consensus.cc:504] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } }
I20260809 19:28:49.557219 6803 leader_election.cc:288] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [CANDIDATE]: Term 1 election: Requested vote from peers dc44732627e0464c940c34e404b5008b (127.21.81.194:39347), 783324c00a734f879546e2d6421d5023 (127.21.81.193:46599)
I20260809 19:28:49.557364 6596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "491364c292f7440f8f2670a4fda1895d" candidate_uuid: "62a76f556e7d472b858da8d7bfb75cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc44732627e0464c940c34e404b5008b"
I20260809 19:28:49.557365 6452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "491364c292f7440f8f2670a4fda1895d" candidate_uuid: "62a76f556e7d472b858da8d7bfb75cfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "783324c00a734f879546e2d6421d5023"
I20260809 19:28:49.557440 6596 raft_consensus.cc:3037] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:49.557461 6452 raft_consensus.cc:3037] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:28:49.558171 6596 raft_consensus.cc:2454] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a76f556e7d472b858da8d7bfb75cfa in term 1.
I20260809 19:28:49.558341 6665 leader_election.cc:302] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a76f556e7d472b858da8d7bfb75cfa, dc44732627e0464c940c34e404b5008b; no voters:
I20260809 19:28:49.558346 6452 raft_consensus.cc:2454] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a76f556e7d472b858da8d7bfb75cfa in term 1.
I20260809 19:28:49.558446 6803 raft_consensus.cc:2781] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:28:49.558615 6803 raft_consensus.cc:686] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [term 1 LEADER]: Becoming Leader. State: Replica: 62a76f556e7d472b858da8d7bfb75cfa, State: Running, Role: LEADER
I20260809 19:28:49.558720 6803 consensus_queue.cc:237] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } }
I20260809 19:28:49.559554 6300 catalog_manager.cc:5495] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa reported cstate change: term changed from 0 to 1, leader changed from <none> to 62a76f556e7d472b858da8d7bfb75cfa (127.21.81.195). New cstate: current_term: 1 leader_uuid: "62a76f556e7d472b858da8d7bfb75cfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } health_report { overall_health: UNKNOWN } } }
I20260809 19:28:49.567287 6796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.565127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56602 (local address 127.21.81.195:39165)
0809 19:28:49.565162 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:28:49.565164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:28:49.565171 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:28:49.565219 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:28:49.565220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:28:49.565236 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:28:49.565294 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:28:49.565386 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.565888 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.566672 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:28:49.566854 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:28:49.566904 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:28:49.567016 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:28:49.567119 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:28:49.567198 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 19:28:49.567219 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 19:28:49.567236 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:28:49.568997 6596 raft_consensus.cc:1261] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b [term 1 FOLLOWER]: Refusing update from remote peer 62a76f556e7d472b858da8d7bfb75cfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:28:49.569062 6452 raft_consensus.cc:1261] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Refusing update from remote peer 62a76f556e7d472b858da8d7bfb75cfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:28:49.569360 6805 consensus_queue.cc:1035] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:28:49.569499 6803 consensus_queue.cc:1035] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:28:49.571448 6808 mvcc.cc:205] Tried to move back new op lower bound from 7316700076312096768 to 7316700076273315840. Current Snapshot: MvccSnapshot[applied={T|T < 7316700076312096768}]
I20260809 19:28:49.571911 6811 mvcc.cc:205] Tried to move back new op lower bound from 7316700076312096768 to 7316700076273315840. Current Snapshot: MvccSnapshot[applied={T|T < 7316700076312096768}]
I20260809 19:28:49.572113 6809 mvcc.cc:205] Tried to move back new op lower bound from 7316700076312096768 to 7316700076273315840. Current Snapshot: MvccSnapshot[applied={T|T < 7316700076312096768}]
I20260809 19:28:49.573915 6301 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:28:49.574013 6301 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:28:49.574721 6301 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Sending DeleteTablet for 3 replicas of tablet 491364c292f7440f8f2670a4fda1895d
I20260809 19:28:49.575119 6575 tablet_service.cc:1514] Processing DeleteTablet for tablet 491364c292f7440f8f2670a4fda1895d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:49 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:44168
I20260809 19:28:49.575192 6431 tablet_service.cc:1514] Processing DeleteTablet for tablet 491364c292f7440f8f2670a4fda1895d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:49 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:60986
I20260809 19:28:49.575220 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 6340
I20260809 19:28:49.575275 6507 tablet_replica.cc:330] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023: stopping tablet replica
I20260809 19:28:49.575323 6507 raft_consensus.cc:2229] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:28:49.575376 6507 raft_consensus.cc:2258] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:28:49.577406 6819 tablet_replica.cc:330] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b: stopping tablet replica
I20260809 19:28:49.577534 6819 raft_consensus.cc:2229] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:28:49.577667 6819 raft_consensus.cc:2258] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:28:49.578102 6819 ts_tablet_manager.cc:1905] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:28:49.579771 6819 ts_tablet_manager.cc:1918] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:28:49.579862 6819 log.cc:1196] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/491364c292f7440f8f2670a4fda1895d
I20260809 19:28:49.580150 6819 ts_tablet_manager.cc:1939] T 491364c292f7440f8f2670a4fda1895d P dc44732627e0464c940c34e404b5008b: Deleting consensus metadata
I20260809 19:28:49.580575 6285 catalog_manager.cc:4841] TS dc44732627e0464c940c34e404b5008b (127.21.81.194:39347): tablet 491364c292f7440f8f2670a4fda1895d (table test-table [id=4e5411353d95456ca570aaf7ffa63a88]) successfully deleted
I20260809 19:28:49.580804 6716 tablet_service.cc:1514] Processing DeleteTablet for tablet 491364c292f7440f8f2670a4fda1895d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:28:49 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:56596
I20260809 19:28:49.581034 6820 tablet_replica.cc:330] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa: stopping tablet replica
I20260809 19:28:49.581166 6820 raft_consensus.cc:2229] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:28:49.581274 6820 raft_consensus.cc:2258] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:28:49.581748 6820 ts_tablet_manager.cc:1905] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:28:49.583065 6820 ts_tablet_manager.cc:1918] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:28:49.583150 6820 log.cc:1196] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/491364c292f7440f8f2670a4fda1895d
I20260809 19:28:49.583470 6820 ts_tablet_manager.cc:1939] T 491364c292f7440f8f2670a4fda1895d P 62a76f556e7d472b858da8d7bfb75cfa: Deleting consensus metadata
I20260809 19:28:49.583855 6287 catalog_manager.cc:4841] TS 62a76f556e7d472b858da8d7bfb75cfa (127.21.81.195:39165): tablet 491364c292f7440f8f2670a4fda1895d (table test-table [id=4e5411353d95456ca570aaf7ffa63a88]) successfully deleted
W20260809 19:28:49.585981 6287 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:46599 (error 108)
I20260809 19:28:49.586390 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 6504
I20260809 19:28:49.586480 6793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:28:49.586342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:46599 (local address 127.0.0.1:60992)
0809 19:28:49.586391 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 19:28:49.586442 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:46599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 19:28:49.588061 6287 catalog_manager.cc:4584] TS 783324c00a734f879546e2d6421d5023 (127.21.81.193:46599): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 491364c292f7440f8f2670a4fda1895d: Network error: Client connection negotiation failed: client connection to 127.21.81.193:46599: connect: Connection refused (error 111)
I20260809 19:28:49.602574 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 6647
I20260809 19:28:49.608479 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 6265
I20260809 19:29:09.616160 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39717
--webserver_interface=127.21.81.254
--webserver_port=46309
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:39717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:09.697990 6822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:09.698179 6822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:09.698200 6822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:09.699338 6822 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 19:29:09.699373 6822 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 19:29:09.699385 6822 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 19:29:09.699397 6822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:09.699446 6822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:09.699470 6822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:09.699496 6822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:09.699539 6822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:09.700946 6822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:39717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=46309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:09.701205 6822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:09.701436 6822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:09.706418 6831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:09.706423 6822 server_base.cc:1034] running on GCE node
W20260809 19:29:09.706434 6828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:09.706429 6829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:09.706852 6822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:09.707136 6822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:09.708294 6822 hybrid_clock.cc:648] HybridClock initialized: now 1786303749708279 us; error 42 us; skew 500 ppm
Aug 09 19:29:09 dist-test-slave-njxd krb5kdc[5667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303749, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:09.710649 6822 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:29:09.711081 6822 webserver.cc:460] Webserver started at http://127.21.81.254:46309/ using document root <none> and password file <none>
I20260809 19:29:09.711242 6822 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:09.711279 6822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:09.712411 6822 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:29:09.712985 6839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:09.713184 6822 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:29:09.713254 6822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "b13d62f274a24b66923d5daaf5ba7b9a"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "102d37588b3fcc67bb4c84b0f3887f1f"
server_key_iv: "9e7e279984ed654778d2907f121f7419"
server_key_version: "encryptionkey@0"
I20260809 19:29:09.713542 6822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:09.724201 6822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:09.725924 6822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:09.726027 6822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:09.730048 6822 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:39717
I20260809 19:29:09.730079 6894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:39717 every 8 connection(s)
I20260809 19:29:09.730468 6822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:09.731737 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 6822
I20260809 19:29:09.731945 6895 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:29:09.732051 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:46599
--local_ip_for_outbound_sockets=127.21.81.193
--tserver_master_addrs=127.21.81.254:39717
--webserver_port=40363
--webserver_interface=127.21.81.193
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:29:09.732981 6895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Bootstrap starting.
I20260809 19:29:09.742429 6895 log.cc:824] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:09.744647 6895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:29:09.744946 6895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Bootstrap complete.
I20260809 19:29:09.747295 6895 raft_consensus.cc:348] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } }
I20260809 19:29:09.747506 6895 raft_consensus.cc:729] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13d62f274a24b66923d5daaf5ba7b9a, State: Initialized, Role: FOLLOWER
I20260809 19:29:09.747668 6895 consensus_queue.cc:260] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } }
I20260809 19:29:09.747733 6895 raft_consensus.cc:388] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:09.747792 6895 raft_consensus.cc:482] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:09.747860 6895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 2 FOLLOWER]: Advancing to term 3
I20260809 19:29:09.749189 6895 raft_consensus.cc:504] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } }
I20260809 19:29:09.749359 6895 leader_election.cc:302] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b13d62f274a24b66923d5daaf5ba7b9a; no voters:
I20260809 19:29:09.749549 6895 leader_election.cc:288] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 19:29:09.749601 6900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 19:29:09.749723 6900 raft_consensus.cc:686] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [term 3 LEADER]: Becoming Leader. State: Replica: b13d62f274a24b66923d5daaf5ba7b9a, State: Running, Role: LEADER
I20260809 19:29:09.749819 6900 consensus_queue.cc:237] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } }
I20260809 19:29:09.749846 6895 sys_catalog.cc:564] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:09.750093 6900 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b13d62f274a24b66923d5daaf5ba7b9a. Latest consensus state: current_term: 3 leader_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } } }
I20260809 19:29:09.750191 6900 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:09.750509 6905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:09.750095 6901 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13d62f274a24b66923d5daaf5ba7b9a" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39717 } } }
I20260809 19:29:09.751253 6901 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:09.751338 6905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:09.751443 6905 catalog_manager.cc:1259] Loaded cluster ID: 29735a69af56418688e2234ced76f6d5
I20260809 19:29:09.751717 6905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:09.752470 6905 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:09.752645 6905 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Loaded TSK: 0
I20260809 19:29:09.752696 6905 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b13d62f274a24b66923d5daaf5ba7b9a: Loaded TSK: 1
F20260809 19:29:09.752925 6905 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3db38ff9ad google::LogMessage::Fail() at ??:0
@ 0x7f3db390288a google::LogMessage::SendToLog() at ??:0
@ 0x7f3db38ff4de google::LogMessage::Flush() at ??:0
@ 0x7f3db3900a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3db606c104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3db6069d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f3dbb8591cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f3dbb85948e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f3dbb859594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f3dbb81fc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f3dbb829ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3db57cd0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3db57c2954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3db66266db start_thread at ??:0
@ 0x7f3db23fd61f clone at ??:0
W20260809 19:29:09.846216 6898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:09.846500 6898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:09.846537 6898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:09.848336 6898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:09.848402 6898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:09.848445 6898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:29:09.850608 6898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:46599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=40363
--tserver_master_addrs=127.21.81.254:39717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:09.850911 6898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:09.851167 6898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:09.856360 6942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:09.856382 6943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:09.856562 6898 server_base.cc:1034] running on GCE node
W20260809 19:29:09.856606 6945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:09.856791 6898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:09.857033 6898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:09.858181 6898 hybrid_clock.cc:648] HybridClock initialized: now 1786303749858170 us; error 32 us; skew 500 ppm
Aug 09 19:29:09 dist-test-slave-njxd krb5kdc[5667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303749, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:09.860846 6898 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:29:09.861394 6898 webserver.cc:460] Webserver started at http://127.21.81.193:40363/ using document root <none> and password file <none>
I20260809 19:29:09.861698 6898 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:09.861768 6898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:09.863492 6898 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:29:09.864300 6953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:09.864516 6898 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:29:09.864627 6898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "783324c00a734f879546e2d6421d5023"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "33e9ad57d392ac2473afab2a1dda6e2b"
server_key_iv: "1b015aea426213b1c97b5db38c0658f4"
server_key_version: "encryptionkey@0"
I20260809 19:29:09.865073 6898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:09.918591 6898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:09.921201 6898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:09.921365 6898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:09.921671 6898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:09.922201 6960 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:29:09.923760 6898 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:29:09.923828 6898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:29:09.923866 6898 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:29:09.924844 6898 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:29:09.924892 6898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 19:29:09.924974 6960 tablet_bootstrap.cc:492] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023: Bootstrap starting.
I20260809 19:29:09.932461 6898 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:46599
I20260809 19:29:09.932706 7076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:46599 every 8 connection(s)
I20260809 19:29:09.932960 6898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:29:09.936594 6960 log.cc:824] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:09.938876 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 6898
I20260809 19:29:09.939054 6960 tablet_bootstrap.cc:492] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:29:09.939203 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:39347
--local_ip_for_outbound_sockets=127.21.81.194
--tserver_master_addrs=127.21.81.254:39717
--webserver_port=41037
--webserver_interface=127.21.81.194
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:29:09.939368 6960 tablet_bootstrap.cc:492] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023: Bootstrap complete.
I20260809 19:29:09.940634 6960 ts_tablet_manager.cc:1397] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.011s
I20260809 19:29:09.941856 6960 raft_consensus.cc:348] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } }
I20260809 19:29:09.942109 6960 raft_consensus.cc:729] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783324c00a734f879546e2d6421d5023, State: Initialized, Role: FOLLOWER
I20260809 19:29:09.943710 6960 consensus_queue.cc:260] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } }
I20260809 19:29:09.944039 6960 ts_tablet_manager.cc:1428] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
W20260809 19:29:09.987211 7080 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:29:09.933724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:51571)
0809 19:29:09.933902 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:09.933949 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:09.934728 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:09.986998 (+ 52270us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
W20260809 19:29:09.987481 7077 heartbeater.cc:644] Failed to heartbeat to 127.21.81.254:39717 (0 consecutive failures): Network error: Failed to ping master at 127.21.81.254:39717: Client connection negotiation failed: client connection to 127.21.81.254:39717: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 19:29:09.987874 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:09.987746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:42539)
0809 19:29:09.987798 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:09.987847 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 19:29:09.988243 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:09.988135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:43721)
0809 19:29:09.988181 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:09.988208 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 19:29:09.988369 7077 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:29:10.025784 7082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:10.025972 7082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:10.025995 7082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:10.027179 7082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:10.027222 7082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:10.027271 7082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:29:10.028710 7082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:39347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=41037
--tserver_master_addrs=127.21.81.254:39717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:10.028940 7082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:10.029168 7082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:10.034191 7089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:10.034198 7088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:10.034205 7091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:10.034430 7082 server_base.cc:1034] running on GCE node
I20260809 19:29:10.034648 7082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:10.034914 7082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:10.036094 7082 hybrid_clock.cc:648] HybridClock initialized: now 1786303750036083 us; error 32 us; skew 500 ppm
Aug 09 19:29:10 dist-test-slave-njxd krb5kdc[5667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303750, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:10.038611 7082 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:29:10.039060 7082 webserver.cc:460] Webserver started at http://127.21.81.194:41037/ using document root <none> and password file <none>
I20260809 19:29:10.039295 7082 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:10.039361 7082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:10.040660 7082 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:29:10.041244 7098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:10.041414 7082 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:10.041512 7082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "dc44732627e0464c940c34e404b5008b"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "68beaa04316269858fe1ebde27236a2f"
server_key_iv: "d68fccb93408d5a628d4eb9f1b13583d"
server_key_version: "encryptionkey@0"
I20260809 19:29:10.041890 7082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:10.048547 7082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:10.050343 7082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:10.050494 7082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:10.050710 7082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:10.050997 7082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:10.051054 7082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:10.051095 7082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:10.051123 7082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:10.057238 7082 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:39347
I20260809 19:29:10.057286 7216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:39347 every 8 connection(s)
I20260809 19:29:10.057680 7082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:29:10.058687 7219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:10.058278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:43297)
0809 19:29:10.058430 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:10.058529 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
W20260809 19:29:10.059063 7217 heartbeater.cc:644] Failed to heartbeat to 127.21.81.254:39717 (0 consecutive failures): Network error: Failed to ping master at 127.21.81.254:39717: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
I20260809 19:29:10.059325 7219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:10.059232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:51183)
0809 19:29:10.059277 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:10.059301 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 19:29:10.059585 7219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:10.059499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:43131)
0809 19:29:10.059543 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:10.059565 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 19:29:10.059685 7217 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 19:29:10.065412 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 7082
I20260809 19:29:10.065845 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:39165
--local_ip_for_outbound_sockets=127.21.81.195
--tserver_master_addrs=127.21.81.254:39717
--webserver_port=34653
--webserver_interface=127.21.81.195
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:10.149109 7220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:10.149334 7220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:10.149371 7220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:10.150519 7220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:10.150584 7220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:10.150628 7220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:29:10.152105 7220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:39165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=34653
--tserver_master_addrs=127.21.81.254:39717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:10.152419 7220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:10.152648 7220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:10.157534 7229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:10.157559 7227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:10.157558 7226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:10.157757 7220 server_base.cc:1034] running on GCE node
I20260809 19:29:10.157945 7220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:10.158215 7220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:10.159370 7220 hybrid_clock.cc:648] HybridClock initialized: now 1786303750159353 us; error 42 us; skew 500 ppm
Aug 09 19:29:10 dist-test-slave-njxd krb5kdc[5667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303750, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:10.161837 7220 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:29:10.162211 7220 webserver.cc:460] Webserver started at http://127.21.81.195:34653/ using document root <none> and password file <none>
I20260809 19:29:10.162402 7220 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:10.162443 7220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:10.163638 7220 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:29:10.164215 7236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:10.164393 7220 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:10.164467 7220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "62a76f556e7d472b858da8d7bfb75cfa"
format_stamp: "Formatted at 2026-08-09 19:28:28 on dist-test-slave-njxd"
server_key: "0872068d888ed7cd1c50c8d9eedc12ab"
server_key_iv: "bdc381a68edc73beeafbda3f59ba724f"
server_key_version: "encryptionkey@0"
I20260809 19:29:10.164755 7220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:10.192914 7220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:10.194742 7220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:10.194897 7220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:10.195112 7220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:10.195436 7220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:10.195470 7220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:10.195492 7220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:10.195536 7220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:10.201381 7220 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:39165
I20260809 19:29:10.201432 7355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:39165 every 8 connection(s)
I20260809 19:29:10.201807 7220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:29:10.202811 7358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:10.202452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:50177)
0809 19:29:10.202600 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:10.202668 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":73,"threads_started":1}
W20260809 19:29:10.203186 7356 heartbeater.cc:644] Failed to heartbeat to 127.21.81.254:39717 (0 consecutive failures): Network error: Failed to ping master at 127.21.81.254:39717: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
I20260809 19:29:10.203442 7358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:10.203358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:41203)
0809 19:29:10.203400 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:10.203426 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 19:29:10.203665 7358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:10.203616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:33811)
0809 19:29:10.203636 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:10.203653 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260809 19:29:10.203773 7356 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 19:29:10.211944 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 7220
I20260809 19:29:10.989066 7360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:10.988797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:35233)
0809 19:29:10.988953 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:10.988996 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260809 19:29:11.060235 7361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:11.059994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:44435)
0809 19:29:11.060138 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:11.060175 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
I20260809 19:29:11.204437 7362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:11.204198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:44163)
0809 19:29:11.204342 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:11.204383 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260809 19:29:11.745877 7363 raft_consensus.cc:482] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:29:11.746011 7363 raft_consensus.cc:504] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } }
I20260809 19:29:11.746320 7363 leader_election.cc:288] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc44732627e0464c940c34e404b5008b (127.21.81.194:39347), 62a76f556e7d472b858da8d7bfb75cfa (127.21.81.195:39165)
Aug 09 19:29:11 dist-test-slave-njxd krb5kdc[5667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303749, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.194@KRBTEST.COM
Aug 09 19:29:11 dist-test-slave-njxd krb5kdc[5667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303749, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.195@KRBTEST.COM
I20260809 19:29:11.764057 7366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:11.746683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:49561 (local address 127.21.81.194:39347)
0809 19:29:11.746846 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:11.746860 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:11.746934 (+ 74us) server_negotiation.cc:407] Connection header received
0809 19:29:11.747514 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:11.747518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:11.747579 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:11.747748 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:11.748899 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:11.750384 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:11.751906 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:11.752568 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:11.752688 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:11.760375 (+ 7687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:11.760380 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:11.760380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:11.760393 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:11.762138 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:11.762658 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:11.762661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:11.762661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:11.762752 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:11.763062 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:11.763063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:11.763063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:11.763336 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:11.763429 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:11.763745 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 19:29:11.763878 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260809 19:29:11.764189 7367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:11.746694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:49255 (local address 127.21.81.195:39165)
0809 19:29:11.746900 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:11.746912 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:11.746933 (+ 21us) server_negotiation.cc:407] Connection header received
0809 19:29:11.747514 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:11.747517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:11.747579 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:11.747747 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:11.748902 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:11.750384 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:11.751906 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:11.752568 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:11.752688 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:11.760645 (+ 7957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:11.760649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:11.760649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:11.760661 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:11.762717 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:11.763067 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:11.763070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:11.763070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:11.763164 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:11.763375 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:11.763376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:11.763377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:11.763617 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:11.763706 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:11.763902 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 19:29:11.764082 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":72,"threads_started":1}
I20260809 19:29:11.764122 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:11.746533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:39165 (local address 127.21.81.193:49255)
0809 19:29:11.746702 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:11.746719 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:11.746798 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:11.747849 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:11.747852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:11.748438 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:11.748751 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:11.748759 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:11.750514 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:11.750516 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:11.751802 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:11.751805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:11.752684 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:11.752684 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:11.752824 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:11.757968 (+ 5144us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:11.757982 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:11.760508 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:11.762807 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:11.762809 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:11.762809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:11.762982 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:11.763239 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:11.763240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:11.763240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:11.763309 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:11.763689 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:11.763690 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:11.763745 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 19:29:11.763835 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 19:29:11.763957 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":37,"thread_start_us":81,"threads_started":1}
I20260809 19:29:11.764101 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:11.746533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:39347 (local address 127.21.81.193:49561)
0809 19:29:11.746867 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:11.746876 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:11.746936 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:11.747850 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:11.747852 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:11.748438 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:11.748751 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:11.748759 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:11.750516 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:11.750516 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:11.751802 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:11.751805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:11.752683 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:11.752684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:11.752824 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:11.757968 (+ 5144us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:11.757982 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:11.760191 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:11.762233 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:11.762244 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:11.762246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:11.762569 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:11.762845 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:11.762847 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:11.762847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:11.762980 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:11.763423 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:11.763431 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:11.763576 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 19:29:11.763687 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 19:29:11.763880 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":111,"thread_start_us":60,"threads_started":1}
W20260809 19:29:11.765398 6955 leader_election.cc:341] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dc44732627e0464c940c34e404b5008b (127.21.81.194:39347): Not found: Tablet not found: 491364c292f7440f8f2670a4fda1895d
W20260809 19:29:11.765491 6957 leader_election.cc:341] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 62a76f556e7d472b858da8d7bfb75cfa (127.21.81.195:39165): Not found: Tablet not found: 491364c292f7440f8f2670a4fda1895d
I20260809 19:29:11.765538 6957 leader_election.cc:302] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 783324c00a734f879546e2d6421d5023; no voters: 62a76f556e7d472b858da8d7bfb75cfa, dc44732627e0464c940c34e404b5008b
I20260809 19:29:11.765686 7363 raft_consensus.cc:2726] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:29:11.989815 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:11.989660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:44159)
0809 19:29:11.989726 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:11.989773 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 19:29:12.060969 7368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:12.060695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:45427)
0809 19:29:12.060879 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:12.060925 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":104,"threads_started":1}
I20260809 19:29:12.205036 7369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:12.204830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:38593)
0809 19:29:12.204949 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:12.204992 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260809 19:29:12.990495 7370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:12.990280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:45589)
0809 19:29:12.990408 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:12.990447 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260809 19:29:13.061652 7372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:13.061405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:50305)
0809 19:29:13.061527 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:13.061562 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260809 19:29:13.205883 7373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:13.205674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:46047)
0809 19:29:13.205805 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:13.205844 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":61,"threads_started":1}
I20260809 19:29:13.653453 7374 raft_consensus.cc:482] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:29:13.653555 7374 raft_consensus.cc:504] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } }
I20260809 19:29:13.653854 7374 leader_election.cc:288] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc44732627e0464c940c34e404b5008b (127.21.81.194:39347), 62a76f556e7d472b858da8d7bfb75cfa (127.21.81.195:39165)
W20260809 19:29:13.654350 6955 leader_election.cc:341] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dc44732627e0464c940c34e404b5008b (127.21.81.194:39347): Not found: Tablet not found: 491364c292f7440f8f2670a4fda1895d
W20260809 19:29:13.654417 6957 leader_election.cc:341] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 62a76f556e7d472b858da8d7bfb75cfa (127.21.81.195:39165): Not found: Tablet not found: 491364c292f7440f8f2670a4fda1895d
I20260809 19:29:13.654520 6957 leader_election.cc:302] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 783324c00a734f879546e2d6421d5023; no voters: 62a76f556e7d472b858da8d7bfb75cfa, dc44732627e0464c940c34e404b5008b
I20260809 19:29:13.654619 7374 raft_consensus.cc:2726] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:29:13.991215 7375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:13.990941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:44403)
0809 19:29:13.991121 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:13.991160 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":79,"threads_started":1}
I20260809 19:29:14.062359 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:14.062094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:53583)
0809 19:29:14.062260 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:14.062302 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":68,"threads_started":1}
I20260809 19:29:14.206564 7377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:14.206354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:44717)
0809 19:29:14.206490 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:14.206528 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260809 19:29:14.991925 7378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:14.991699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:54527)
0809 19:29:14.991847 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:14.991877 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260809 19:29:15.063028 7379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:15.062830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:46131)
0809 19:29:15.062946 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:15.062987 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260809 19:29:15.207405 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:15.207077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:58383)
0809 19:29:15.207313 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:15.207357 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":67,"threads_started":1}
I20260809 19:29:15.600634 7381 raft_consensus.cc:482] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:29:15.600744 7381 raft_consensus.cc:504] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } }
I20260809 19:29:15.600989 7381 leader_election.cc:288] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc44732627e0464c940c34e404b5008b (127.21.81.194:39347), 62a76f556e7d472b858da8d7bfb75cfa (127.21.81.195:39165)
W20260809 19:29:15.601354 6955 leader_election.cc:341] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dc44732627e0464c940c34e404b5008b (127.21.81.194:39347): Not found: Tablet not found: 491364c292f7440f8f2670a4fda1895d
W20260809 19:29:15.601436 6957 leader_election.cc:341] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 62a76f556e7d472b858da8d7bfb75cfa (127.21.81.195:39165): Not found: Tablet not found: 491364c292f7440f8f2670a4fda1895d
I20260809 19:29:15.601485 6957 leader_election.cc:302] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 783324c00a734f879546e2d6421d5023; no voters: 62a76f556e7d472b858da8d7bfb75cfa, dc44732627e0464c940c34e404b5008b
I20260809 19:29:15.601559 7381 raft_consensus.cc:2726] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:29:15.992568 7382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:15.992356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:60343)
0809 19:29:15.992480 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:15.992519 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260809 19:29:16.063777 7383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:16.063548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:50075)
0809 19:29:16.063696 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:16.063735 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260809 19:29:16.208182 7384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:16.207980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:39359)
0809 19:29:16.208104 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:16.208141 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260809 19:29:16.993268 7385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:16.993037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:41709)
0809 19:29:16.993179 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:16.993222 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260809 19:29:17.064420 7386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:17.064214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:51747)
0809 19:29:17.064341 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:17.064377 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260809 19:29:17.208895 7387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:17.208673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:58377)
0809 19:29:17.208817 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:17.208852 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
I20260809 19:29:17.993944 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:17.993739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:46433)
0809 19:29:17.993859 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:17.993895 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260809 19:29:18.065016 7389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:18.064806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:37755)
0809 19:29:18.064940 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:18.064973 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260809 19:29:18.209878 7390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:18.209497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:35733)
0809 19:29:18.209784 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:18.209830 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":149,"threads_started":1}
I20260809 19:29:18.994606 7391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:18.994355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:55667)
0809 19:29:18.994509 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:18.994545 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
I20260809 19:29:19.065653 7392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:19.065444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:41899)
0809 19:29:19.065555 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:19.065608 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
I20260809 19:29:19.210562 7393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:19.210324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:59081)
0809 19:29:19.210459 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:19.210497 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260809 19:29:19.995383 7394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:19.995164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:42801)
0809 19:29:19.995301 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:19.995334 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260809 19:29:20.052556 7395 raft_consensus.cc:482] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:29:20.052650 7395 raft_consensus.cc:504] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc44732627e0464c940c34e404b5008b" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39347 } } peers { permanent_uuid: "62a76f556e7d472b858da8d7bfb75cfa" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 39165 } } peers { permanent_uuid: "783324c00a734f879546e2d6421d5023" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46599 } }
I20260809 19:29:20.052831 7395 leader_election.cc:288] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc44732627e0464c940c34e404b5008b (127.21.81.194:39347), 62a76f556e7d472b858da8d7bfb75cfa (127.21.81.195:39165)
W20260809 19:29:20.053206 6955 leader_election.cc:341] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dc44732627e0464c940c34e404b5008b (127.21.81.194:39347): Not found: Tablet not found: 491364c292f7440f8f2670a4fda1895d
W20260809 19:29:20.053299 6957 leader_election.cc:341] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 62a76f556e7d472b858da8d7bfb75cfa (127.21.81.195:39165): Not found: Tablet not found: 491364c292f7440f8f2670a4fda1895d
I20260809 19:29:20.053373 6957 leader_election.cc:302] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 783324c00a734f879546e2d6421d5023; no voters: 62a76f556e7d472b858da8d7bfb75cfa, dc44732627e0464c940c34e404b5008b
I20260809 19:29:20.053483 7395 raft_consensus.cc:2726] T 491364c292f7440f8f2670a4fda1895d P 783324c00a734f879546e2d6421d5023 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:29:20.066215 7396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:20.066008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:55585)
0809 19:29:20.066134 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:20.066168 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260809 19:29:20.211254 7398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:20.211008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:32983)
0809 19:29:20.211158 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:20.211201 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":62,"threads_started":1}
I20260809 19:29:20.996065 7399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:20.995829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:49501)
0809 19:29:20.995965 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:20.996003 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260809 19:29:21.066880 7400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:21.066643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:52423)
0809 19:29:21.066791 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:21.066837 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":61,"threads_started":1}
I20260809 19:29:21.211988 7401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:21.211751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:57755)
0809 19:29:21.211903 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:21.211941 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
I20260809 19:29:21.996778 7402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:21.996538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:46889)
0809 19:29:21.996677 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:21.996728 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260809 19:29:22.067544 7403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:22.067319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:53833)
0809 19:29:22.067463 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:22.067500 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260809 19:29:22.212780 7404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:22.212473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:50371)
0809 19:29:22.212686 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:22.212729 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":64,"threads_started":1}
I20260809 19:29:22.997424 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:22.997210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:44565)
0809 19:29:22.997337 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:22.997378 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260809 19:29:23.068174 7406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:23.067962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:44455)
0809 19:29:23.068099 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:23.068134 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260809 19:29:23.213681 7407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:23.213375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:56107)
0809 19:29:23.213531 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:23.213625 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":64,"threads_started":1}
I20260809 19:29:23.998144 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:23.997900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:40781)
0809 19:29:23.998038 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:23.998085 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":59,"threads_started":1}
I20260809 19:29:24.068885 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:24.068645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:45331)
0809 19:29:24.068798 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:24.068837 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
I20260809 19:29:24.214519 7410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:24.214279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:45693)
0809 19:29:24.214422 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:24.214462 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260809 19:29:24.998817 7411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:24.998617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.193:37107)
0809 19:29:24.998741 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:24.998772 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260809 19:29:25.069847 7412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:25.069527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.194:35671)
0809 19:29:25.069736 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:25.069771 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":125,"threads_started":1}
I20260809 19:29:25.215236 7413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:25.215015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39717 (local address 127.21.81.195:58963)
0809 19:29:25.215151 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:25.215190 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.254:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260809 19:29:25.218588 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 6898
I20260809 19:29:25.228534 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 7082
I20260809 19:29:25.233916 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 7220
2026-08-09T19:29:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58255 ms)
[----------] 30 tests from SecurityITest (239777 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:25 dist-test-slave-njxd krb5kdc[7416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:25 dist-test-slave-njxd krb5kdc[7416](info): set up 2 sockets
Aug 09 19:29:25 dist-test-slave-njxd krb5kdc[7416](info): commencing operation
krb5kdc: starting...
W20260809 19:29:26.308965 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:26 dist-test-slave-njxd krb5kdc[7416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:29:26.433255 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:34343
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:46617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:34343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:26.515897 7429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:26.516144 7429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:26.516185 7429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:26.517355 7429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:26.517448 7429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:26.517474 7429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:26.517504 7429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:26.517539 7429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:26.519055 7429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:34343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:34343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:26.519332 7429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:26.519562 7429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:26.524785 7440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:26.524808 7437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:26.524837 7429 server_base.cc:1034] running on GCE node
W20260809 19:29:26.524785 7438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:26.525224 7429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:26.525563 7429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:26.526763 7429 hybrid_clock.cc:648] HybridClock initialized: now 1786303766526743 us; error 37 us; skew 500 ppm
Aug 09 19:29:26 dist-test-slave-njxd krb5kdc[7416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303766, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:26.529374 7429 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:29:26.529887 7429 webserver.cc:460] Webserver started at http://127.21.81.254:41373/ using document root <none> and password file <none>
I20260809 19:29:26.530117 7429 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:26.530189 7429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:26.530342 7429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:26.531409 7429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "f33a18b25c47465e9fbb2ecd35190f0b"
format_stamp: "Formatted at 2026-08-09 19:29:26 on dist-test-slave-njxd"
server_key: "0bd06bbfe9f006e38cd5375fe4f49a6e"
server_key_iv: "a9ec97d5cdb8fc13fa8684d4aa00c422"
server_key_version: "encryptionkey@0"
I20260809 19:29:26.531785 7429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "f33a18b25c47465e9fbb2ecd35190f0b"
format_stamp: "Formatted at 2026-08-09 19:29:26 on dist-test-slave-njxd"
server_key: "0bd06bbfe9f006e38cd5375fe4f49a6e"
server_key_iv: "a9ec97d5cdb8fc13fa8684d4aa00c422"
server_key_version: "encryptionkey@0"
I20260809 19:29:26.533160 7429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:29:26.534001 7447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:26.534176 7429 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:26.534286 7429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "f33a18b25c47465e9fbb2ecd35190f0b"
format_stamp: "Formatted at 2026-08-09 19:29:26 on dist-test-slave-njxd"
server_key: "0bd06bbfe9f006e38cd5375fe4f49a6e"
server_key_iv: "a9ec97d5cdb8fc13fa8684d4aa00c422"
server_key_version: "encryptionkey@0"
I20260809 19:29:26.534363 7429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:26.566287 7429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:26.568792 7429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:26.568969 7429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:26.573311 7429 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:34343
I20260809 19:29:26.573362 7501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:34343 every 8 connection(s)
I20260809 19:29:26.573802 7429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:26.574714 7502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:26.577068 7502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b: Bootstrap starting.
I20260809 19:29:26.577754 7502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:26.578052 7502 log.cc:824] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:26.578785 7502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b: No bootstrap required, opened a new log
I20260809 19:29:26.579444 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 7429
I20260809 19:29:26.579550 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:26.579757 21831 external_mini_cluster.cc:1428] Setting key 21fa4195c3da2cc9a6ff1d75cedeb044
I20260809 19:29:26.581068 7502 raft_consensus.cc:348] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f33a18b25c47465e9fbb2ecd35190f0b" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34343 } }
I20260809 19:29:26.581246 7502 raft_consensus.cc:374] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:26.581328 7502 raft_consensus.cc:729] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f33a18b25c47465e9fbb2ecd35190f0b, State: Initialized, Role: FOLLOWER
I20260809 19:29:26.581496 7502 consensus_queue.cc:260] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f33a18b25c47465e9fbb2ecd35190f0b" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34343 } }
I20260809 19:29:26.581617 7502 raft_consensus.cc:388] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:26.581656 7502 raft_consensus.cc:482] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:26.581718 7502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:26.582700 7502 raft_consensus.cc:504] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f33a18b25c47465e9fbb2ecd35190f0b" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34343 } }
I20260809 19:29:26.582796 7502 leader_election.cc:302] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f33a18b25c47465e9fbb2ecd35190f0b; no voters:
I20260809 19:29:26.582965 7502 leader_election.cc:288] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:26.583017 7507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:26.583211 7507 raft_consensus.cc:686] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [term 1 LEADER]: Becoming Leader. State: Replica: f33a18b25c47465e9fbb2ecd35190f0b, State: Running, Role: LEADER
I20260809 19:29:26.583271 7502 sys_catalog.cc:564] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:26.583384 7507 consensus_queue.cc:237] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f33a18b25c47465e9fbb2ecd35190f0b" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34343 } }
I20260809 19:29:26.583868 7509 sys_catalog.cc:455] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f33a18b25c47465e9fbb2ecd35190f0b. Latest consensus state: current_term: 1 leader_uuid: "f33a18b25c47465e9fbb2ecd35190f0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f33a18b25c47465e9fbb2ecd35190f0b" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34343 } } }
I20260809 19:29:26.583900 7508 sys_catalog.cc:455] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f33a18b25c47465e9fbb2ecd35190f0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f33a18b25c47465e9fbb2ecd35190f0b" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34343 } } }
I20260809 19:29:26.584033 7508 sys_catalog.cc:458] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:26.584257 7509 sys_catalog.cc:458] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:26.584288 7516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:26.584964 7516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:26.586822 7516 catalog_manager.cc:1347] Generated new cluster ID: 31343581b5164a5d90a51e2406797590
I20260809 19:29:26.586877 7516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:29:26 dist-test-slave-njxd krb5kdc[7416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:29:26.591641 7506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:26.580290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50146 (local address 127.21.81.254:34343)
0809 19:29:26.580490 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:26.580498 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:26.580524 (+ 26us) server_negotiation.cc:407] Connection header received
0809 19:29:26.581047 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:26.581053 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:26.581106 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:26.581219 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:26.582071 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:26.583122 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:26.584238 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:26.584874 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:26.585556 (+ 682us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:26.587817 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:26.587820 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:26.587820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:26.587828 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:26.589491 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:26.589921 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:26.589924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:26.589925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:26.590033 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:26.590249 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:26.590250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:26.590251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:26.590528 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:26.591314 (+ 786us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:26.591381 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 19:29:26.591483 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260809 19:29:26.608657 7516 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:26.609205 7516 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:26.612820 7516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b: Generated new TSK 0
I20260809 19:29:26.612995 7516 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:29:26.644912 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:34343
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:46617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:26.728673 7529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:26.728899 7529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:26.728924 7529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:26.730268 7529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:26.730315 7529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:26.730365 7529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:29:26.731823 7529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:34343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:26.732076 7529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:26.732311 7529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:26.737388 7538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:26.737370 7536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:26.737417 7535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:26.737694 7529 server_base.cc:1034] running on GCE node
I20260809 19:29:26.737897 7529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:26.738152 7529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:26.739300 7529 hybrid_clock.cc:648] HybridClock initialized: now 1786303766739283 us; error 35 us; skew 500 ppm
Aug 09 19:29:26 dist-test-slave-njxd krb5kdc[7416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303766, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:26.741942 7529 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:29:26.742380 7529 webserver.cc:460] Webserver started at http://127.21.81.193:41523/ using document root <none> and password file <none>
I20260809 19:29:26.742568 7529 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:26.742605 7529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:26.742693 7529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:26.743682 7529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "60d6f700e7d947619ef2430ae14a1d9d"
format_stamp: "Formatted at 2026-08-09 19:29:26 on dist-test-slave-njxd"
server_key: "7bfcd2d433cab1616087e5c635ef9697"
server_key_iv: "7ad06e230747a5634beba0a180fe2611"
server_key_version: "encryptionkey@0"
I20260809 19:29:26.744014 7529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "60d6f700e7d947619ef2430ae14a1d9d"
format_stamp: "Formatted at 2026-08-09 19:29:26 on dist-test-slave-njxd"
server_key: "7bfcd2d433cab1616087e5c635ef9697"
server_key_iv: "7ad06e230747a5634beba0a180fe2611"
server_key_version: "encryptionkey@0"
I20260809 19:29:26.745296 7529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:29:26.746080 7545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:26.746387 7529 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:29:26.746497 7529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "60d6f700e7d947619ef2430ae14a1d9d"
format_stamp: "Formatted at 2026-08-09 19:29:26 on dist-test-slave-njxd"
server_key: "7bfcd2d433cab1616087e5c635ef9697"
server_key_iv: "7ad06e230747a5634beba0a180fe2611"
server_key_version: "encryptionkey@0"
I20260809 19:29:26.746574 7529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:26.768272 7529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:26.770707 7529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:26.770903 7529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:26.771158 7529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:26.771476 7529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:26.771531 7529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:26.771574 7529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:26.771616 7529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:26.777732 7529 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:32785
I20260809 19:29:26.777782 7662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:32785 every 8 connection(s)
I20260809 19:29:26.778092 7529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:29:26.780833 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 7529
I20260809 19:29:26.780936 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:29:26.781129 21831 external_mini_cluster.cc:1428] Setting key 51d6f8fe19e09b4b4aadcfec1fc5bcbd
Aug 09 19:29:26 dist-test-slave-njxd krb5kdc[7416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303766, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:29:26.788103 7506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:26.778798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:47873 (local address 127.21.81.254:34343)
0809 19:29:26.778873 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:26.778875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:26.779086 (+ 211us) server_negotiation.cc:407] Connection header received
0809 19:29:26.779549 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:26.779550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:26.779560 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:26.779612 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:26.780520 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:26.781288 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:26.782364 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:26.782595 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:26.782694 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:26.785180 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:26.785182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:26.785182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:26.785189 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:26.786641 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:26.787080 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:26.787082 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:26.787082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:26.787149 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:26.787425 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:26.787425 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:26.787426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:26.787545 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:26.787617 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:26.787910 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 19:29:26.788011 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:29:26.788296 7666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:26.778780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:34343 (local address 127.21.81.193:47873)
0809 19:29:26.778959 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:26.778996 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:26.779463 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:26.779664 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:26.779667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:26.780243 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:26.780439 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:26.780444 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:26.781376 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:26.781377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:26.782268 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:26.782271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:26.782703 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:26.782704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:26.782836 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:26.783359 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:26.783367 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:26.785079 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:26.786754 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:26.786763 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:26.786765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:26.786997 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:26.787219 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:26.787220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:26.787220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:26.787346 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:26.787621 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:26.787630 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:26.787771 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 19:29:26.787884 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 19:29:26.788052 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
I20260809 19:29:26.788843 7664 heartbeater.cc:347] Connected to a master server at 127.21.81.254:34343
I20260809 19:29:26.788940 7664 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:26.789150 7664 heartbeater.cc:511] Master 127.21.81.254:34343 requested a full tablet report, sending...
I20260809 19:29:26.789795 7464 ts_manager.cc:194] Registered new tserver with Master: 60d6f700e7d947619ef2430ae14a1d9d (127.21.81.193:32785)
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
I20260809 19:29:26.790747 7464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:47873
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:29:26.808475 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:34343
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:46617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:26.890641 7670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:26.890849 7670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:26.890873 7670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:26.891957 7670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:26.891990 7670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:26.892016 7670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:29:26.893467 7670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:34343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:26.893725 7670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:26.893920 7670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:26.899051 7675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:26.899107 7678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:26.899086 7676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:26.899134 7670 server_base.cc:1034] running on GCE node
I20260809 19:29:26.899376 7670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:26.899585 7670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:26.900723 7670 hybrid_clock.cc:648] HybridClock initialized: now 1786303766900715 us; error 32 us; skew 500 ppm
Aug 09 19:29:26 dist-test-slave-njxd krb5kdc[7416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303766, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:26.903275 7670 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:29:26.903700 7670 webserver.cc:460] Webserver started at http://127.21.81.194:35219/ using document root <none> and password file <none>
I20260809 19:29:26.903883 7670 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:26.903923 7670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:26.904016 7670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:26.905035 7670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "d1f5dfa951f7421f93710e0850ed98ec"
format_stamp: "Formatted at 2026-08-09 19:29:26 on dist-test-slave-njxd"
server_key: "2720dfaf58e1cd41e4f4da3afbe33c32"
server_key_iv: "d07ca719144446f88c42aee6cdb11fc8"
server_key_version: "encryptionkey@0"
I20260809 19:29:26.905344 7670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "d1f5dfa951f7421f93710e0850ed98ec"
format_stamp: "Formatted at 2026-08-09 19:29:26 on dist-test-slave-njxd"
server_key: "2720dfaf58e1cd41e4f4da3afbe33c32"
server_key_iv: "d07ca719144446f88c42aee6cdb11fc8"
server_key_version: "encryptionkey@0"
I20260809 19:29:26.906706 7670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:29:26.907411 7685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:26.907565 7670 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:26.907655 7670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "d1f5dfa951f7421f93710e0850ed98ec"
format_stamp: "Formatted at 2026-08-09 19:29:26 on dist-test-slave-njxd"
server_key: "2720dfaf58e1cd41e4f4da3afbe33c32"
server_key_iv: "d07ca719144446f88c42aee6cdb11fc8"
server_key_version: "encryptionkey@0"
I20260809 19:29:26.907712 7670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:26.919651 7670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:26.921435 7670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:26.921541 7670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:26.921811 7670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:26.922068 7670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:26.922098 7670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:26.922119 7670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:26.922132 7670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:26.927834 7670 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:39295
I20260809 19:29:26.927892 7806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:39295 every 8 connection(s)
I20260809 19:29:26.928231 7670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:29:26.933629 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 7670
I20260809 19:29:26.933744 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:29:26.933972 21831 external_mini_cluster.cc:1428] Setting key 0d0af58572cbe76bcedef010d1c91618
Aug 09 19:29:26 dist-test-slave-njxd krb5kdc[7416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303766, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:29:26.939774 7506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:26.928968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:51413 (local address 127.21.81.254:34343)
0809 19:29:26.929049 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:26.929052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:26.929215 (+ 163us) server_negotiation.cc:407] Connection header received
0809 19:29:26.929774 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:26.929776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:26.929787 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:26.929843 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:26.930735 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:26.931266 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:26.932177 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:26.932350 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:26.932399 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:26.936566 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:26.936567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:26.936568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:26.936574 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:26.938380 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:26.938784 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:26.938785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:26.938786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:26.938853 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:26.939088 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:26.939089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:26.939089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:26.939257 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:26.939334 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:26.939581 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 19:29:26.939680 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 19:29:26.939971 7810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:26.928910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:34343 (local address 127.21.81.194:51413)
0809 19:29:26.929098 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:26.929134 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:26.929666 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:26.929914 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:26.929917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:26.930459 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:26.930659 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:26.930663 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:26.931359 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:26.931359 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:26.932105 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:26.932106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:26.932421 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:26.932422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:26.932523 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:26.932966 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:26.932974 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:26.934670 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:26.938461 (+ 3791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:26.938469 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:26.938471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:26.938699 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:26.938920 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:26.938921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:26.938921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:26.939020 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:26.939326 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:26.939331 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:26.939443 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 19:29:26.939553 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 19:29:26.939719 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
I20260809 19:29:26.940537 7807 heartbeater.cc:347] Connected to a master server at 127.21.81.254:34343
I20260809 19:29:26.940621 7807 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:26.940809 7807 heartbeater.cc:511] Master 127.21.81.254:34343 requested a full tablet report, sending...
I20260809 19:29:26.941231 7464 ts_manager.cc:194] Registered new tserver with Master: d1f5dfa951f7421f93710e0850ed98ec (127.21.81.194:39295)
I20260809 19:29:26.941692 7464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:51413
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:29:26.960539 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:34343
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:46617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:27.045065 7813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:27.045298 7813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:27.045349 7813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:27.046532 7813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:27.046577 7813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:27.046603 7813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:29:27.048100 7813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:34343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:27.048370 7813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:27.048656 7813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:27.053874 7819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:27.053993 7813 server_base.cc:1034] running on GCE node
W20260809 19:29:27.053886 7821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:27.053898 7818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:27.054302 7813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:27.054597 7813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:27.055769 7813 hybrid_clock.cc:648] HybridClock initialized: now 1786303767055754 us; error 30 us; skew 500 ppm
Aug 09 19:29:27 dist-test-slave-njxd krb5kdc[7416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303767, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:27.058445 7813 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:29:27.058950 7813 webserver.cc:460] Webserver started at http://127.21.81.195:42111/ using document root <none> and password file <none>
I20260809 19:29:27.059182 7813 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:27.059250 7813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:27.059403 7813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:27.060463 7813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "26052f61f4c34ed488f6fd7d4683e00e"
format_stamp: "Formatted at 2026-08-09 19:29:27 on dist-test-slave-njxd"
server_key: "b4b532729c93cd61f44133b59176bf00"
server_key_iv: "cc91f210ad59e4d87467807f06a3805a"
server_key_version: "encryptionkey@0"
I20260809 19:29:27.060875 7813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "26052f61f4c34ed488f6fd7d4683e00e"
format_stamp: "Formatted at 2026-08-09 19:29:27 on dist-test-slave-njxd"
server_key: "b4b532729c93cd61f44133b59176bf00"
server_key_iv: "cc91f210ad59e4d87467807f06a3805a"
server_key_version: "encryptionkey@0"
I20260809 19:29:27.062296 7813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 19:29:27.063045 7829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:27.063248 7813 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:27.063356 7813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "26052f61f4c34ed488f6fd7d4683e00e"
format_stamp: "Formatted at 2026-08-09 19:29:27 on dist-test-slave-njxd"
server_key: "b4b532729c93cd61f44133b59176bf00"
server_key_iv: "cc91f210ad59e4d87467807f06a3805a"
server_key_version: "encryptionkey@0"
I20260809 19:29:27.063432 7813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:27.083508 7813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:27.085983 7813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:27.086148 7813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:27.086395 7813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:27.086716 7813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:27.086753 7813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:27.086807 7813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:27.086834 7813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:27.092954 7813 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:44473
I20260809 19:29:27.093029 7947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:44473 every 8 connection(s)
I20260809 19:29:27.093415 7813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:29:27.096161 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 7813
I20260809 19:29:27.096242 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:29:27.096426 21831 external_mini_cluster.cc:1428] Setting key 9e9f1858b6b9e74bde6b199fbb5c952a
Aug 09 19:29:27 dist-test-slave-njxd krb5kdc[7416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303767, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:29:27.102998 7506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.094243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:48485 (local address 127.21.81.254:34343)
0809 19:29:27.094303 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.094305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.094535 (+ 230us) server_negotiation.cc:407] Connection header received
0809 19:29:27.095091 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.095092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.095106 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.095158 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:27.096134 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.096853 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.098030 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.098242 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.098327 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.100397 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:27.100398 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:27.100398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:27.100405 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:27.101787 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:27.102156 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:27.102157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:27.102157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:27.102207 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:27.102423 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:27.102423 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:27.102424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:27.102524 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:27.102585 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.102863 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.102932 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:29:27.103194 7950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.094219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:34343 (local address 127.21.81.195:48485)
0809 19:29:27.094412 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:27.094450 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:27.094973 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:27.095237 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:27.095241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:27.095825 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:27.096021 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.096025 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.096944 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.096945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.097911 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.097914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.098341 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.098342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.098467 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.098925 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:27.098934 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:27.100311 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:27.101860 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:27.101867 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:27.101868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:27.102083 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:27.102268 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:27.102268 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:27.102269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:27.102369 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:27.102588 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:27.102592 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:27.102746 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 19:29:27.102832 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 19:29:27.102951 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260809 19:29:27.103700 7948 heartbeater.cc:347] Connected to a master server at 127.21.81.254:34343
I20260809 19:29:27.103797 7948 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:27.103961 7948 heartbeater.cc:511] Master 127.21.81.254:34343 requested a full tablet report, sending...
I20260809 19:29:27.104326 7464 ts_manager.cc:194] Registered new tserver with Master: 26052f61f4c34ed488f6fd7d4683e00e (127.21.81.195:44473)
I20260809 19:29:27.104714 7464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:48485
I20260809 19:29:27.108107 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:29:27.115658 7506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.109421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50162 (local address 127.21.81.254:34343)
0809 19:29:27.109482 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.109484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.109622 (+ 138us) server_negotiation.cc:407] Connection header received
0809 19:29:27.109729 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.109730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.109741 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.109790 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:27.110342 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.110842 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.111620 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.111855 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.111931 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.113222 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:27.113223 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:27.113224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:27.113232 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:27.114452 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:27.114823 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:27.114825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:27.114825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:27.114885 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:27.115091 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:27.115092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:27.115092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:27.115253 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:27.115361 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.115475 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.115559 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 19:29:27.117143 7464 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:29:27.118913 7464 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:29:27.125455 7960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.121295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:32785 (local address 127.0.0.1:56926)
0809 19:29:27.121667 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:27.121678 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:27.121735 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:27.122107 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:27.122110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:27.122113 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:27.122192 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.122194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.123061 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.123062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.124182 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.124185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.125132 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.125133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.125211 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.125224 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:29:27.125307 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:29:27.125365 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":99,"threads_started":1}
I20260809 19:29:27.125479 7963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.121503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56926 (local address 127.21.81.193:32785)
0809 19:29:27.121762 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.121767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.121779 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:29:27.121819 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.121821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.121835 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.122005 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:27.122273 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.122959 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.124273 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.125037 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.125112 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.125177 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.125338 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.125384 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":118,"threads_started":1}
I20260809 19:29:27.125671 7964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.121652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49642 (local address 127.21.81.194:39295)
0809 19:29:27.121834 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.121838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.121906 (+ 68us) server_negotiation.cc:407] Connection header received
0809 19:29:27.121985 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.121987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.122004 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.122080 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:27.122300 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.123438 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.124600 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.125119 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.125189 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.125247 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.125500 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.125551 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":108,"threads_started":1}
I20260809 19:29:27.125979 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.121291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:44473 (local address 127.0.0.1:60070)
0809 19:29:27.121464 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:27.121506 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:27.121571 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:27.122242 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:27.122243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:27.122245 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:27.122282 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.122284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.123604 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.123605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.124711 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.124713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.125731 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.125732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.125798 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.125803 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:29:27.125861 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 19:29:27.125899 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":66,"threads_started":1}
I20260809 19:29:27.125989 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.121403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:39295 (local address 127.0.0.1:49642)
0809 19:29:27.121854 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:27.121863 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:27.121917 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:27.122161 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:27.122162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:27.122165 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:27.122218 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.122220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.123554 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.123555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.124510 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.124512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.125282 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.125283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.125338 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.125341 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:29:27.125386 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:29:27.125418 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":68,"threads_started":1}
I20260809 19:29:27.126359 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.121462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60070 (local address 127.21.81.195:44473)
0809 19:29:27.122067 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.122071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.122080 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:29:27.122118 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.122120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.122144 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.122357 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:27.122436 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.123485 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.124812 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.125543 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.125705 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.125767 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.126109 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.126275 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":93,"threads_started":1}
I20260809 19:29:27.126703 7595 tablet_service.cc:1467] Processing CreateTablet for tablet 9220714fc4594d6581e7555dee921bea (DEFAULT_TABLE table=test-table [id=3e242ab3a5754423a3f5e4e8607fee22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:27.126991 7595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9220714fc4594d6581e7555dee921bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:27.127074 7878 tablet_service.cc:1467] Processing CreateTablet for tablet 9220714fc4594d6581e7555dee921bea (DEFAULT_TABLE table=test-table [id=3e242ab3a5754423a3f5e4e8607fee22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:27.127446 7878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9220714fc4594d6581e7555dee921bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:27.129765 7965 tablet_bootstrap.cc:492] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d: Bootstrap starting.
I20260809 19:29:27.130316 7966 tablet_bootstrap.cc:492] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e: Bootstrap starting.
I20260809 19:29:27.130542 7965 tablet_bootstrap.cc:654] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:27.130453 7736 tablet_service.cc:1467] Processing CreateTablet for tablet 9220714fc4594d6581e7555dee921bea (DEFAULT_TABLE table=test-table [id=3e242ab3a5754423a3f5e4e8607fee22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:27.130811 7736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9220714fc4594d6581e7555dee921bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:27.131042 7966 tablet_bootstrap.cc:654] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:27.131172 7965 log.cc:824] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:27.131606 7966 log.cc:824] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:27.132010 7965 tablet_bootstrap.cc:492] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d: No bootstrap required, opened a new log
I20260809 19:29:27.132056 7965 ts_tablet_manager.cc:1397] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:29:27.132411 7966 tablet_bootstrap.cc:492] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e: No bootstrap required, opened a new log
I20260809 19:29:27.132465 7966 ts_tablet_manager.cc:1397] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:29:27.133559 7969 tablet_bootstrap.cc:492] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec: Bootstrap starting.
I20260809 19:29:27.134280 7969 tablet_bootstrap.cc:654] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:27.134173 7965 raft_consensus.cc:348] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d6f700e7d947619ef2430ae14a1d9d" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 32785 } } peers { permanent_uuid: "26052f61f4c34ed488f6fd7d4683e00e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44473 } } peers { permanent_uuid: "d1f5dfa951f7421f93710e0850ed98ec" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39295 } }
I20260809 19:29:27.134351 7965 raft_consensus.cc:374] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:27.134428 7965 raft_consensus.cc:729] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60d6f700e7d947619ef2430ae14a1d9d, State: Initialized, Role: FOLLOWER
I20260809 19:29:27.134616 7965 consensus_queue.cc:260] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d6f700e7d947619ef2430ae14a1d9d" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 32785 } } peers { permanent_uuid: "26052f61f4c34ed488f6fd7d4683e00e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44473 } } peers { permanent_uuid: "d1f5dfa951f7421f93710e0850ed98ec" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39295 } }
I20260809 19:29:27.134548 7966 raft_consensus.cc:348] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d6f700e7d947619ef2430ae14a1d9d" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 32785 } } peers { permanent_uuid: "26052f61f4c34ed488f6fd7d4683e00e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44473 } } peers { permanent_uuid: "d1f5dfa951f7421f93710e0850ed98ec" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39295 } }
I20260809 19:29:27.134747 7966 raft_consensus.cc:374] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:27.134820 7966 raft_consensus.cc:729] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26052f61f4c34ed488f6fd7d4683e00e, State: Initialized, Role: FOLLOWER
I20260809 19:29:27.134927 7965 ts_tablet_manager.cc:1428] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:29:27.134920 7969 log.cc:824] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:27.134943 7966 consensus_queue.cc:260] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d6f700e7d947619ef2430ae14a1d9d" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 32785 } } peers { permanent_uuid: "26052f61f4c34ed488f6fd7d4683e00e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44473 } } peers { permanent_uuid: "d1f5dfa951f7421f93710e0850ed98ec" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39295 } }
I20260809 19:29:27.135051 7664 heartbeater.cc:503] Master 127.21.81.254:34343 was elected leader, sending a full tablet report...
I20260809 19:29:27.135212 7966 ts_tablet_manager.cc:1428] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:29:27.135324 7948 heartbeater.cc:503] Master 127.21.81.254:34343 was elected leader, sending a full tablet report...
I20260809 19:29:27.135869 7969 tablet_bootstrap.cc:492] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec: No bootstrap required, opened a new log
I20260809 19:29:27.135948 7969 ts_tablet_manager.cc:1397] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:29:27.137482 7969 raft_consensus.cc:348] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d6f700e7d947619ef2430ae14a1d9d" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 32785 } } peers { permanent_uuid: "26052f61f4c34ed488f6fd7d4683e00e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44473 } } peers { permanent_uuid: "d1f5dfa951f7421f93710e0850ed98ec" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39295 } }
I20260809 19:29:27.137657 7969 raft_consensus.cc:374] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:27.137701 7969 raft_consensus.cc:729] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1f5dfa951f7421f93710e0850ed98ec, State: Initialized, Role: FOLLOWER
I20260809 19:29:27.137804 7969 consensus_queue.cc:260] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d6f700e7d947619ef2430ae14a1d9d" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 32785 } } peers { permanent_uuid: "26052f61f4c34ed488f6fd7d4683e00e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44473 } } peers { permanent_uuid: "d1f5dfa951f7421f93710e0850ed98ec" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39295 } }
I20260809 19:29:27.138029 7969 ts_tablet_manager.cc:1428] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:29:27.138092 7807 heartbeater.cc:503] Master 127.21.81.254:34343 was elected leader, sending a full tablet report...
W20260809 19:29:27.178874 7808 tablet.cc:2369] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:29:27.278889 7665 tablet.cc:2369] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:29:27.323673 7973 raft_consensus.cc:482] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:29:27.323824 7973 raft_consensus.cc:504] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d6f700e7d947619ef2430ae14a1d9d" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 32785 } } peers { permanent_uuid: "26052f61f4c34ed488f6fd7d4683e00e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44473 } } peers { permanent_uuid: "d1f5dfa951f7421f93710e0850ed98ec" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39295 } }
I20260809 19:29:27.324198 7973 leader_election.cc:288] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60d6f700e7d947619ef2430ae14a1d9d (127.21.81.193:32785), 26052f61f4c34ed488f6fd7d4683e00e (127.21.81.195:44473)
I20260809 19:29:27.327214 7963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.324514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:53371 (local address 127.21.81.193:32785)
0809 19:29:27.324634 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.324645 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.324656 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:29:27.324695 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.324696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.324707 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.324789 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:27.324959 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.325429 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.326453 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.326832 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.326930 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.326939 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.327107 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.327146 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 19:29:27.327214 7810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.324420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:32785 (local address 127.21.81.194:53371)
0809 19:29:27.324489 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:27.324501 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:27.324550 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:27.324795 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:27.324796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:27.324798 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:27.324877 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.324879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.325518 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.325519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.326347 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.326349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.326929 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.326930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.327020 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.327023 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:29:27.327100 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 19:29:27.327143 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 19:29:27.327275 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.324549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:33519 (local address 127.21.81.195:44473)
0809 19:29:27.324631 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.324633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.324765 (+ 132us) server_negotiation.cc:407] Connection header received
0809 19:29:27.324827 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.324828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.324839 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.324917 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:27.325085 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.325544 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.326525 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.326913 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.327003 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.327012 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.327163 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.327202 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 19:29:27.327353 7974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.324458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:44473 (local address 127.21.81.194:33519)
0809 19:29:27.324679 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:27.324691 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:27.324769 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:27.324924 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:27.324925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:27.324926 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:27.324999 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.325001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.325700 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.325701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.326429 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.326431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.327004 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.327005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.327077 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.327081 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:29:27.327157 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 19:29:27.327196 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":142,"threads_started":1}
I20260809 19:29:27.327623 7616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9220714fc4594d6581e7555dee921bea" candidate_uuid: "d1f5dfa951f7421f93710e0850ed98ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60d6f700e7d947619ef2430ae14a1d9d" is_pre_election: true
I20260809 19:29:27.327713 7899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9220714fc4594d6581e7555dee921bea" candidate_uuid: "d1f5dfa951f7421f93710e0850ed98ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26052f61f4c34ed488f6fd7d4683e00e" is_pre_election: true
I20260809 19:29:27.327806 7616 raft_consensus.cc:2454] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1f5dfa951f7421f93710e0850ed98ec in term 0.
I20260809 19:29:27.327891 7899 raft_consensus.cc:2454] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1f5dfa951f7421f93710e0850ed98ec in term 0.
I20260809 19:29:27.328066 7688 leader_election.cc:302] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60d6f700e7d947619ef2430ae14a1d9d, d1f5dfa951f7421f93710e0850ed98ec; no voters:
I20260809 19:29:27.328275 7973 raft_consensus.cc:2781] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:29:27.328347 7973 raft_consensus.cc:482] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:29:27.328378 7973 raft_consensus.cc:3037] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:27.329108 7973 raft_consensus.cc:504] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d6f700e7d947619ef2430ae14a1d9d" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 32785 } } peers { permanent_uuid: "26052f61f4c34ed488f6fd7d4683e00e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44473 } } peers { permanent_uuid: "d1f5dfa951f7421f93710e0850ed98ec" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39295 } }
I20260809 19:29:27.329270 7973 leader_election.cc:288] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [CANDIDATE]: Term 1 election: Requested vote from peers 60d6f700e7d947619ef2430ae14a1d9d (127.21.81.193:32785), 26052f61f4c34ed488f6fd7d4683e00e (127.21.81.195:44473)
I20260809 19:29:27.329452 7899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9220714fc4594d6581e7555dee921bea" candidate_uuid: "d1f5dfa951f7421f93710e0850ed98ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26052f61f4c34ed488f6fd7d4683e00e"
I20260809 19:29:27.329526 7899 raft_consensus.cc:3037] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:27.329646 7616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9220714fc4594d6581e7555dee921bea" candidate_uuid: "d1f5dfa951f7421f93710e0850ed98ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60d6f700e7d947619ef2430ae14a1d9d"
I20260809 19:29:27.329754 7616 raft_consensus.cc:3037] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:27.330403 7899 raft_consensus.cc:2454] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1f5dfa951f7421f93710e0850ed98ec in term 1.
I20260809 19:29:27.330479 7616 raft_consensus.cc:2454] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1f5dfa951f7421f93710e0850ed98ec in term 1.
I20260809 19:29:27.330637 7686 leader_election.cc:302] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26052f61f4c34ed488f6fd7d4683e00e, d1f5dfa951f7421f93710e0850ed98ec; no voters:
I20260809 19:29:27.330730 7973 raft_consensus.cc:2781] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:27.330922 7973 raft_consensus.cc:686] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [term 1 LEADER]: Becoming Leader. State: Replica: d1f5dfa951f7421f93710e0850ed98ec, State: Running, Role: LEADER
I20260809 19:29:27.331032 7973 consensus_queue.cc:237] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d6f700e7d947619ef2430ae14a1d9d" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 32785 } } peers { permanent_uuid: "26052f61f4c34ed488f6fd7d4683e00e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44473 } } peers { permanent_uuid: "d1f5dfa951f7421f93710e0850ed98ec" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39295 } }
I20260809 19:29:27.331907 7464 catalog_manager.cc:5495] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec reported cstate change: term changed from 0 to 1, leader changed from <none> to d1f5dfa951f7421f93710e0850ed98ec (127.21.81.194). New cstate: current_term: 1 leader_uuid: "d1f5dfa951f7421f93710e0850ed98ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d6f700e7d947619ef2430ae14a1d9d" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 32785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26052f61f4c34ed488f6fd7d4683e00e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1f5dfa951f7421f93710e0850ed98ec" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 39295 } health_report { overall_health: HEALTHY } } }
W20260809 19:29:27.344062 7949 tablet.cc:2369] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:29:27.353385 7964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.351130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49652 (local address 127.21.81.194:39295)
0809 19:29:27.351195 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.351197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.351206 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:29:27.351274 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.351274 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.351291 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.351350 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:29:27.351480 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.351938 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.352681 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.352862 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.352911 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.353044 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:29:27.353137 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:29:27.353194 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.353241 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.353258 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 19:29:27.355269 7899 raft_consensus.cc:1261] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e [term 1 FOLLOWER]: Refusing update from remote peer d1f5dfa951f7421f93710e0850ed98ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:29:27.355269 7616 raft_consensus.cc:1261] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d [term 1 FOLLOWER]: Refusing update from remote peer d1f5dfa951f7421f93710e0850ed98ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:29:27.355657 7975 consensus_queue.cc:1035] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "60d6f700e7d947619ef2430ae14a1d9d" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 32785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:29:27.355804 7973 consensus_queue.cc:1035] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "26052f61f4c34ed488f6fd7d4683e00e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 44473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:29:27.359217 7978 mvcc.cc:205] Tried to move back new op lower bound from 7316700231084335104 to 7316700230988443648. Current Snapshot: MvccSnapshot[applied={T|T < 7316700231084335104 or (T in {7316700231084335104})}]
I20260809 19:29:27.359344 7980 mvcc.cc:205] Tried to move back new op lower bound from 7316700231084335104 to 7316700230988443648. Current Snapshot: MvccSnapshot[applied={T|T < 7316700231084335104 or (T in {7316700231084335104})}]
I20260809 19:29:27.360394 7463 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:29:27.360495 7463 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:29:27.361268 7463 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f33a18b25c47465e9fbb2ecd35190f0b: Sending DeleteTablet for 3 replicas of tablet 9220714fc4594d6581e7555dee921bea
I20260809 19:29:27.361625 7595 tablet_service.cc:1514] Processing DeleteTablet for tablet 9220714fc4594d6581e7555dee921bea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:27 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:56926
I20260809 19:29:27.361629 7878 tablet_service.cc:1514] Processing DeleteTablet for tablet 9220714fc4594d6581e7555dee921bea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:27 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:60070
I20260809 19:29:27.361708 7736 tablet_service.cc:1514] Processing DeleteTablet for tablet 9220714fc4594d6581e7555dee921bea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:27 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:49642
I20260809 19:29:27.361889 7990 tablet_replica.cc:330] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec: stopping tablet replica
I20260809 19:29:27.361933 7988 tablet_replica.cc:330] T 9220714fc4594d6581e7555dee921bea P 60d6f700e7d947619ef2430ae14a1d9d: stopping tablet replica
I20260809 19:29:27.362027 7990 raft_consensus.cc:2229] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:29:27.362054 7989 tablet_replica.cc:330] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e: stopping tablet replica
I20260809 19:29:27.362149 7990 raft_consensus.cc:2258] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:29:27.362156 7989 raft_consensus.cc:2229] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:29:27.362260 7989 raft_consensus.cc:2258] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:29:27.362409 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 7529
I20260809 19:29:27.362548 7990 ts_tablet_manager.cc:1905] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:29:27.364394 7990 ts_tablet_manager.cc:1918] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:29:27.364454 7990 log.cc:1196] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/9220714fc4594d6581e7555dee921bea
I20260809 19:29:27.364696 7990 ts_tablet_manager.cc:1939] T 9220714fc4594d6581e7555dee921bea P d1f5dfa951f7421f93710e0850ed98ec: Deleting consensus metadata
I20260809 19:29:27.368175 7451 catalog_manager.cc:4841] TS d1f5dfa951f7421f93710e0850ed98ec (127.21.81.194:39295): tablet 9220714fc4594d6581e7555dee921bea (table test-table [id=3e242ab3a5754423a3f5e4e8607fee22]) successfully deleted
I20260809 19:29:27.370143 7989 ts_tablet_manager.cc:1905] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:29:27.371477 7989 ts_tablet_manager.cc:1918] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:29:27.371567 7989 log.cc:1196] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/9220714fc4594d6581e7555dee921bea
I20260809 19:29:27.371853 7989 ts_tablet_manager.cc:1939] T 9220714fc4594d6581e7555dee921bea P 26052f61f4c34ed488f6fd7d4683e00e: Deleting consensus metadata
I20260809 19:29:27.372265 7448 catalog_manager.cc:4841] TS 26052f61f4c34ed488f6fd7d4683e00e (127.21.81.195:44473): tablet 9220714fc4594d6581e7555dee921bea (table test-table [id=3e242ab3a5754423a3f5e4e8607fee22]) successfully deleted
W20260809 19:29:27.374930 7450 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:32785 (error 108)
I20260809 19:29:27.375118 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 7670
I20260809 19:29:27.376408 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.376277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:32785 (local address 127.0.0.1:56936)
0809 19:29:27.376331 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:27.376372 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:32785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 19:29:27.376533 7450 catalog_manager.cc:4584] TS 60d6f700e7d947619ef2430ae14a1d9d (127.21.81.193:32785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9220714fc4594d6581e7555dee921bea: Network error: Client connection negotiation failed: client connection to 127.21.81.193:32785: connect: Connection refused (error 111)
I20260809 19:29:27.383718 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 7813
I20260809 19:29:27.389200 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 7429
2026-08-09T19:29:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2155 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T19:29:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:27Z Disabled control of system clock
I20260809 19:29:27.419039 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:35963
--webserver_interface=127.21.81.254
--webserver_port=0
--builtin_ntp_servers=127.21.81.212:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:35963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:29:27.502053 7995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:27.502290 7995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:27.502311 7995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:27.503511 7995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:27.503554 7995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:27.503567 7995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:27.503580 7995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:27.503592 7995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:27.504886 7995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:35963
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:35963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:27.505086 7995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:27.505331 7995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:27.510315 8001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:27.510416 7995 server_base.cc:1034] running on GCE node
W20260809 19:29:27.510324 8000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:27.510331 8003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:27.510776 7995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:27.511097 7995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:27.512256 7995 hybrid_clock.cc:648] HybridClock initialized: now 1786303767512231 us; error 37 us; skew 500 ppm
I20260809 19:29:27.513752 7995 webserver.cc:460] Webserver started at http://127.21.81.254:43319/ using document root <none> and password file <none>
I20260809 19:29:27.514005 7995 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:27.514055 7995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:27.514222 7995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:27.515347 7995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "7f2f364bc5f049d9a29ba888e40467a0"
format_stamp: "Formatted at 2026-08-09 19:29:27 on dist-test-slave-njxd"
server_key: "619183da60d566503da6c31c1e717258"
server_key_iv: "3cbc0155fdcc767673083f546bab0b26"
server_key_version: "encryptionkey@0"
I20260809 19:29:27.515729 7995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "7f2f364bc5f049d9a29ba888e40467a0"
format_stamp: "Formatted at 2026-08-09 19:29:27 on dist-test-slave-njxd"
server_key: "619183da60d566503da6c31c1e717258"
server_key_iv: "3cbc0155fdcc767673083f546bab0b26"
server_key_version: "encryptionkey@0"
I20260809 19:29:27.517079 7995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:29:27.517895 8010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:27.518051 7995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:29:27.518138 7995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "7f2f364bc5f049d9a29ba888e40467a0"
format_stamp: "Formatted at 2026-08-09 19:29:27 on dist-test-slave-njxd"
server_key: "619183da60d566503da6c31c1e717258"
server_key_iv: "3cbc0155fdcc767673083f546bab0b26"
server_key_version: "encryptionkey@0"
I20260809 19:29:27.518193 7995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:27.525696 7995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:27.525967 7995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:27.526070 7995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:27.530049 7995 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:35963
I20260809 19:29:27.530128 8065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:35963 every 8 connection(s)
I20260809 19:29:27.530452 7995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:27.531463 8066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:27.533506 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 7995
I20260809 19:29:27.533658 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:27.533856 21831 external_mini_cluster.cc:1428] Setting key 4bbba9f04aff4c7a178ce936345b5872
I20260809 19:29:27.534209 8066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0: Bootstrap starting.
I20260809 19:29:27.535235 8066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:27.535585 8066 log.cc:824] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:27.536370 8066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0: No bootstrap required, opened a new log
I20260809 19:29:27.538851 8066 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f364bc5f049d9a29ba888e40467a0" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 35963 } }
I20260809 19:29:27.539048 8066 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:27.539100 8066 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f2f364bc5f049d9a29ba888e40467a0, State: Initialized, Role: FOLLOWER
I20260809 19:29:27.539229 8066 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f364bc5f049d9a29ba888e40467a0" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 35963 } }
I20260809 19:29:27.539327 8066 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:27.539373 8066 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:27.539414 8066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:27.540201 8066 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f364bc5f049d9a29ba888e40467a0" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 35963 } }
I20260809 19:29:27.540285 8069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.534424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59140 (local address 127.21.81.254:35963)
0809 19:29:27.534637 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.534645 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.534679 (+ 34us) server_negotiation.cc:407] Connection header received
0809 19:29:27.535296 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.535306 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.535397 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.535556 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:27.535759 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.537073 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.537945 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.538681 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.538813 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.539518 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:27.539522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:27.539522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:27.539753 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:27.539796 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:27.539812 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:27.539882 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.539944 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.540028 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260809 19:29:27.540314 8066 leader_election.cc:302] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f2f364bc5f049d9a29ba888e40467a0; no voters:
I20260809 19:29:27.540731 8066 leader_election.cc:288] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:27.540778 8072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:27.540971 8072 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [term 1 LEADER]: Becoming Leader. State: Replica: 7f2f364bc5f049d9a29ba888e40467a0, State: Running, Role: LEADER
I20260809 19:29:27.541018 8066 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:27.541075 8072 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f364bc5f049d9a29ba888e40467a0" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 35963 } }
I20260809 19:29:27.541446 8073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f2f364bc5f049d9a29ba888e40467a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f364bc5f049d9a29ba888e40467a0" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 35963 } } }
I20260809 19:29:27.541543 8073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:27.541433 8072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f2f364bc5f049d9a29ba888e40467a0. Latest consensus state: current_term: 1 leader_uuid: "7f2f364bc5f049d9a29ba888e40467a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f364bc5f049d9a29ba888e40467a0" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 35963 } } }
I20260809 19:29:27.541795 8082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:27.542616 8082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:27.542752 8072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:27.544196 8082 catalog_manager.cc:1347] Generated new cluster ID: ef10f957eb9b4e7188e7c14d944517e0
I20260809 19:29:27.544269 8082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:27.555140 8082 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:27.555680 8082 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:27.559978 8082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0: Generated new TSK 0
I20260809 19:29:27.560174 8082 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:27.565429 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:35963
--builtin_ntp_servers=127.21.81.212:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:29:27.657023 8092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:27.657238 8092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:27.657262 8092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:27.658519 8092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:27.658572 8092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:27.658598 8092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:29:27.659993 8092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:35963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:27.660243 8092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:27.660583 8092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:27.666540 8097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:27.666591 8098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:27.666591 8100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:27.666898 8092 server_base.cc:1034] running on GCE node
I20260809 19:29:27.667129 8092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:27.667384 8092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:27.668591 8092 hybrid_clock.cc:648] HybridClock initialized: now 1786303767668546 us; error 40 us; skew 500 ppm
I20260809 19:29:27.670035 8092 webserver.cc:460] Webserver started at http://127.21.81.193:37969/ using document root <none> and password file <none>
I20260809 19:29:27.670303 8092 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:27.670372 8092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:27.670534 8092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:27.671670 8092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "328bfefe32fe490ea90b685743e0e5a3"
format_stamp: "Formatted at 2026-08-09 19:29:27 on dist-test-slave-njxd"
server_key: "d5840146c53392e0472f6d081a9ce8ce"
server_key_iv: "a917d517c98753e644a56ce575af1b90"
server_key_version: "encryptionkey@0"
I20260809 19:29:27.672050 8092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "328bfefe32fe490ea90b685743e0e5a3"
format_stamp: "Formatted at 2026-08-09 19:29:27 on dist-test-slave-njxd"
server_key: "d5840146c53392e0472f6d081a9ce8ce"
server_key_iv: "a917d517c98753e644a56ce575af1b90"
server_key_version: "encryptionkey@0"
I20260809 19:29:27.673482 8092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:29:27.674327 8106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:27.674603 8092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:29:27.674726 8092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "328bfefe32fe490ea90b685743e0e5a3"
format_stamp: "Formatted at 2026-08-09 19:29:27 on dist-test-slave-njxd"
server_key: "d5840146c53392e0472f6d081a9ce8ce"
server_key_iv: "a917d517c98753e644a56ce575af1b90"
server_key_version: "encryptionkey@0"
I20260809 19:29:27.674798 8092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:27.704254 8092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:27.704577 8092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:27.704690 8092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:27.704907 8092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:27.705193 8092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:27.705231 8092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:27.705255 8092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:27.705268 8092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:27.711673 8092 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:42009
I20260809 19:29:27.711750 8225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:42009 every 8 connection(s)
I20260809 19:29:27.712103 8092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:29:27.713181 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 8092
I20260809 19:29:27.713304 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:29:27.713512 21831 external_mini_cluster.cc:1428] Setting key ffae2b6cef19b8ca6d05472230b6c2e4
I20260809 19:29:27.714520 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:35963
--builtin_ntp_servers=127.21.81.212:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 19:29:27.718322 8069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.712891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:43115 (local address 127.21.81.254:35963)
0809 19:29:27.712964 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.712966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.713248 (+ 282us) server_negotiation.cc:407] Connection header received
0809 19:29:27.714024 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.714025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.714041 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.714112 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:27.714990 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.715495 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.716899 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.717093 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.717180 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.718074 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:27.718076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:27.718076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:27.718081 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:27.718096 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:27.718101 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:27.718154 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.718221 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.718246 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 19:29:27.718545 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.712876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:35963 (local address 127.21.81.193:43115)
0809 19:29:27.713119 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:27.713155 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:27.713881 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:27.714188 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:27.714192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:27.714203 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:27.714879 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.714885 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.715617 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.715618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.716805 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.716808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.717191 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.717192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.717331 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.717923 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:29:27.717923 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:27.717952 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:29:27.717973 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:27.718156 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:27.718161 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:27.718164 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:29:27.718227 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 19:29:27.718323 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":118,"threads_started":1}
I20260809 19:29:27.719115 8226 heartbeater.cc:347] Connected to a master server at 127.21.81.254:35963
I20260809 19:29:27.719221 8226 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:27.719398 8226 heartbeater.cc:511] Master 127.21.81.254:35963 requested a full tablet report, sending...
I20260809 19:29:27.719959 8029 ts_manager.cc:194] Registered new tserver with Master: 328bfefe32fe490ea90b685743e0e5a3 (127.21.81.193:42009)
I20260809 19:29:27.720956 8029 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.81.193:43115
W20260809 19:29:27.798484 8229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:27.798705 8229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:27.798728 8229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:27.799858 8229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:27.799892 8229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:27.799916 8229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:29:27.801223 8229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:35963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:27.801414 8229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:27.801680 8229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:27.806761 8235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:27.806746 8237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:27.806764 8234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:27.807005 8229 server_base.cc:1034] running on GCE node
I20260809 19:29:27.807235 8229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:27.807505 8229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:27.808647 8229 hybrid_clock.cc:648] HybridClock initialized: now 1786303767808633 us; error 32 us; skew 500 ppm
I20260809 19:29:27.810048 8229 webserver.cc:460] Webserver started at http://127.21.81.194:34629/ using document root <none> and password file <none>
I20260809 19:29:27.810302 8229 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:27.810351 8229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:27.810519 8229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:27.811655 8229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "6d73232888454dd48104ce7952fcf1d9"
format_stamp: "Formatted at 2026-08-09 19:29:27 on dist-test-slave-njxd"
server_key: "26afa0932f50e37ecf8be4e011ae62f9"
server_key_iv: "833360c73784801beea687ea630acf23"
server_key_version: "encryptionkey@0"
I20260809 19:29:27.812017 8229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "6d73232888454dd48104ce7952fcf1d9"
format_stamp: "Formatted at 2026-08-09 19:29:27 on dist-test-slave-njxd"
server_key: "26afa0932f50e37ecf8be4e011ae62f9"
server_key_iv: "833360c73784801beea687ea630acf23"
server_key_version: "encryptionkey@0"
I20260809 19:29:27.813313 8229 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260809 19:29:27.814091 8243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:27.814270 8229 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:27.814360 8229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "6d73232888454dd48104ce7952fcf1d9"
format_stamp: "Formatted at 2026-08-09 19:29:27 on dist-test-slave-njxd"
server_key: "26afa0932f50e37ecf8be4e011ae62f9"
server_key_iv: "833360c73784801beea687ea630acf23"
server_key_version: "encryptionkey@0"
I20260809 19:29:27.814433 8229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:27.825546 8229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:27.825939 8229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:27.826093 8229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:27.826346 8229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:27.826704 8229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:27.826740 8229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:27.826781 8229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:27.826808 8229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:27.832672 8229 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:40651
I20260809 19:29:27.832774 8362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:40651 every 8 connection(s)
I20260809 19:29:27.833072 8229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:29:27.838171 8069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.833874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:51753 (local address 127.21.81.254:35963)
0809 19:29:27.833937 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.833939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.834166 (+ 227us) server_negotiation.cc:407] Connection header received
0809 19:29:27.834666 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.834667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.834678 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.834731 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:27.835377 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.835868 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.836805 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.836984 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.837032 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.837933 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:27.837935 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:27.837935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:27.837940 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:27.837955 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:27.837959 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:27.838019 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.838084 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.838109 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 19:29:27.838384 8365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.833853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:35963 (local address 127.21.81.194:51753)
0809 19:29:27.834042 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:27.834076 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:27.834557 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:27.834799 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:27.834802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:27.834810 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:27.835308 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.835313 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.835951 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.835952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.836725 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.836726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.837059 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.837060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.837173 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.837792 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:29:27.837792 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:27.837821 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:29:27.837844 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:27.838021 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:27.838023 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:27.838025 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:29:27.838071 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 19:29:27.838146 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260809 19:29:27.838999 8363 heartbeater.cc:347] Connected to a master server at 127.21.81.254:35963
I20260809 19:29:27.839116 8363 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:27.839318 8363 heartbeater.cc:511] Master 127.21.81.254:35963 requested a full tablet report, sending...
I20260809 19:29:27.839712 8029 ts_manager.cc:194] Registered new tserver with Master: 6d73232888454dd48104ce7952fcf1d9 (127.21.81.194:40651)
I20260809 19:29:27.840111 8029 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.81.194:51753
I20260809 19:29:27.840231 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 8229
I20260809 19:29:27.840340 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:29:27.840541 21831 external_mini_cluster.cc:1428] Setting key 0c858ab9057ac954e5a1ceca3b8448d3
I20260809 19:29:27.841485 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:35963
--builtin_ntp_servers=127.21.81.212:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:29:27.924606 8366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:27.924805 8366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:27.924827 8366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:27.926004 8366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:27.926041 8366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:27.926065 8366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:29:27.927310 8366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:35963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:27.927491 8366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:27.927717 8366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:27.932570 8372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:27.932673 8371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:27.932576 8374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:27.932790 8366 server_base.cc:1034] running on GCE node
I20260809 19:29:27.933008 8366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:27.933274 8366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:27.934419 8366 hybrid_clock.cc:648] HybridClock initialized: now 1786303767934401 us; error 35 us; skew 500 ppm
I20260809 19:29:27.935618 8366 webserver.cc:460] Webserver started at http://127.21.81.195:36605/ using document root <none> and password file <none>
I20260809 19:29:27.935871 8366 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:27.935947 8366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:27.936110 8366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:27.937124 8366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "fa16b73aac334b9cb7bbf92182c80872"
format_stamp: "Formatted at 2026-08-09 19:29:27 on dist-test-slave-njxd"
server_key: "43af2402fd30c1970585a07e99e1f077"
server_key_iv: "6eec3e2bcd6d243e87030718095a3e13"
server_key_version: "encryptionkey@0"
I20260809 19:29:27.937487 8366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "fa16b73aac334b9cb7bbf92182c80872"
format_stamp: "Formatted at 2026-08-09 19:29:27 on dist-test-slave-njxd"
server_key: "43af2402fd30c1970585a07e99e1f077"
server_key_iv: "6eec3e2bcd6d243e87030718095a3e13"
server_key_version: "encryptionkey@0"
I20260809 19:29:27.938756 8366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:29:27.939486 8380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:27.939664 8366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:29:27.939764 8366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "fa16b73aac334b9cb7bbf92182c80872"
format_stamp: "Formatted at 2026-08-09 19:29:27 on dist-test-slave-njxd"
server_key: "43af2402fd30c1970585a07e99e1f077"
server_key_iv: "6eec3e2bcd6d243e87030718095a3e13"
server_key_version: "encryptionkey@0"
I20260809 19:29:27.939850 8366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:27.951009 8366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:27.951279 8366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:27.951426 8366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:27.951629 8366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:27.951953 8366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:27.952008 8366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:27.952046 8366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:27.952075 8366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:27.957839 8366 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:42903
I20260809 19:29:27.957887 8498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:42903 every 8 connection(s)
I20260809 19:29:27.958184 8366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:29:27.962993 8069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.958881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:32939 (local address 127.21.81.254:35963)
0809 19:29:27.958963 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.958965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.959145 (+ 180us) server_negotiation.cc:407] Connection header received
0809 19:29:27.959590 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.959592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.959604 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.959660 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:27.960284 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.960789 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.961753 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.961932 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.961984 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.962777 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:27.962778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:27.962778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:27.962782 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:27.962795 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:27.962798 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:27.962839 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.962918 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.962939 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 19:29:27.963223 8502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.958816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:35963 (local address 127.21.81.195:32939)
0809 19:29:27.959033 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:27.959064 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:27.959512 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:27.959724 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:27.959727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:27.959737 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:27.960204 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.960208 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.960871 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.960872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.961671 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.961673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.962009 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.962009 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.962113 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.962657 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:29:27.962658 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:27.962683 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:29:27.962704 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:27.962853 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:27.962857 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:27.962860 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:29:27.962908 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 19:29:27.962980 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
I20260809 19:29:27.963721 8499 heartbeater.cc:347] Connected to a master server at 127.21.81.254:35963
I20260809 19:29:27.963824 8499 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:27.964011 8499 heartbeater.cc:511] Master 127.21.81.254:35963 requested a full tablet report, sending...
I20260809 19:29:27.964365 8029 ts_manager.cc:194] Registered new tserver with Master: fa16b73aac334b9cb7bbf92182c80872 (127.21.81.195:42903)
I20260809 19:29:27.964730 8029 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.81.195:32939
I20260809 19:29:27.966920 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 8366
I20260809 19:29:27.967023 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:29:27.967229 21831 external_mini_cluster.cc:1428] Setting key 69850e28d71aebbd2faf8a54b3cbda5d
I20260809 19:29:27.968096 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:29:27.972579 8069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.969480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59150 (local address 127.21.81.254:35963)
0809 19:29:27.969546 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.969548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.969742 (+ 194us) server_negotiation.cc:407] Connection header received
0809 19:29:27.969789 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.969790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.969816 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.969868 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:27.970024 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.970731 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.971490 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.971789 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.971845 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.972404 (+ 559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:27.972405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:27.972406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:27.972409 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:27.972422 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:27.972425 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:27.972466 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.972511 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.972529 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:29:27.974246 8029 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:29:27.975836 8029 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:29:27.982340 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.978279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39540 (local address 127.21.81.193:42009)
0809 19:29:27.978901 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.978906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.978916 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:29:27.978958 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.978960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.978975 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.979046 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:27.979144 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.979837 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.981149 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.981926 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.982008 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.982108 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.982199 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.982237 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":76,"threads_started":1}
I20260809 19:29:27.982336 8512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.978040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:42903 (local address 127.0.0.1:49336)
0809 19:29:27.978553 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:27.978562 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:27.978600 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:27.978882 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:27.978883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:27.978885 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:27.978928 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.978930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.980237 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.980238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.981232 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.981235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.982105 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.982106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.982172 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.982177 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:29:27.982208 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 19:29:27.982246 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":3,"thread_start_us":293,"threads_started":1}
I20260809 19:29:27.982398 8508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.978030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:42009 (local address 127.0.0.1:39540)
0809 19:29:27.978323 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:27.978362 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:27.978428 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:27.979041 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:27.979042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:27.979044 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:27.979076 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.979077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.979928 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.979929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.981057 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.981059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.982019 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.982020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.982097 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.982108 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 19:29:27.982185 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 19:29:27.982248 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":57,"threads_started":1}
I20260809 19:29:27.982585 8511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.978233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48338 (local address 127.21.81.194:40651)
0809 19:29:27.978575 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.978579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.978595 (+ 16us) server_negotiation.cc:407] Connection header received
0809 19:29:27.978643 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.978645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.978661 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.978735 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:27.978984 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.980216 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.981410 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.982009 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.982077 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.982163 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.982441 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.982484 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":80,"threads_started":1}
I20260809 19:29:27.982739 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.978030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:40651 (local address 127.0.0.1:48338)
0809 19:29:27.978388 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:27.978398 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:27.978447 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:27.978815 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:27.978818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:27.978820 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:27.978898 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.978900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.980302 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.980303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.981326 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:27.981328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.982197 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.982198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:27.982250 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.982253 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:29:27.982320 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 19:29:27.982356 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":48,"thread_start_us":228,"threads_started":1}
I20260809 19:29:27.982790 8509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:27.978135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49336 (local address 127.21.81.195:42903)
0809 19:29:27.978707 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:27.978710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:27.978720 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:29:27.978752 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:27.978754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:27.978767 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:27.979004 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:27.979056 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.980131 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.981329 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:27.981995 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:27.982570 (+ 575us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:27.982658 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:27.982682 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 19:29:27.982715 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":80,"threads_started":1}
I20260809 19:29:27.983448 8429 tablet_service.cc:1467] Processing CreateTablet for tablet 311d057fd3494c99b2a02028a28a2bd0 (DEFAULT_TABLE table=test-table [id=93b04ac790c248e5a2e7921975149ea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:27.983512 8294 tablet_service.cc:1467] Processing CreateTablet for tablet 311d057fd3494c99b2a02028a28a2bd0 (DEFAULT_TABLE table=test-table [id=93b04ac790c248e5a2e7921975149ea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:27.983662 8157 tablet_service.cc:1467] Processing CreateTablet for tablet 311d057fd3494c99b2a02028a28a2bd0 (DEFAULT_TABLE table=test-table [id=93b04ac790c248e5a2e7921975149ea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:27.983856 8294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 311d057fd3494c99b2a02028a28a2bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:27.983937 8429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 311d057fd3494c99b2a02028a28a2bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:27.984200 8157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 311d057fd3494c99b2a02028a28a2bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:27.986471 8514 tablet_bootstrap.cc:492] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9: Bootstrap starting.
I20260809 19:29:27.987079 8514 tablet_bootstrap.cc:654] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:27.987403 8515 tablet_bootstrap.cc:492] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872: Bootstrap starting.
I20260809 19:29:27.987473 8514 log.cc:824] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:27.988046 8515 tablet_bootstrap.cc:654] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:27.988363 8515 log.cc:824] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:27.988441 8514 tablet_bootstrap.cc:492] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9: No bootstrap required, opened a new log
I20260809 19:29:27.988523 8514 ts_tablet_manager.cc:1397] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:29:27.988799 8516 tablet_bootstrap.cc:492] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3: Bootstrap starting.
I20260809 19:29:27.989144 8515 tablet_bootstrap.cc:492] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872: No bootstrap required, opened a new log
I20260809 19:29:27.989229 8515 ts_tablet_manager.cc:1397] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:29:27.989539 8516 tablet_bootstrap.cc:654] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:27.989923 8516 log.cc:824] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:27.990774 8516 tablet_bootstrap.cc:492] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3: No bootstrap required, opened a new log
I20260809 19:29:27.990844 8516 ts_tablet_manager.cc:1397] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:29:27.991304 8514 raft_consensus.cc:348] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d73232888454dd48104ce7952fcf1d9" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 40651 } } peers { permanent_uuid: "328bfefe32fe490ea90b685743e0e5a3" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42009 } } peers { permanent_uuid: "fa16b73aac334b9cb7bbf92182c80872" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42903 } }
I20260809 19:29:27.991487 8514 raft_consensus.cc:374] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:27.991524 8514 raft_consensus.cc:729] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d73232888454dd48104ce7952fcf1d9, State: Initialized, Role: FOLLOWER
I20260809 19:29:27.991444 8515 raft_consensus.cc:348] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d73232888454dd48104ce7952fcf1d9" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 40651 } } peers { permanent_uuid: "328bfefe32fe490ea90b685743e0e5a3" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42009 } } peers { permanent_uuid: "fa16b73aac334b9cb7bbf92182c80872" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42903 } }
I20260809 19:29:27.991631 8515 raft_consensus.cc:374] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:27.991694 8515 raft_consensus.cc:729] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa16b73aac334b9cb7bbf92182c80872, State: Initialized, Role: FOLLOWER
I20260809 19:29:27.991712 8514 consensus_queue.cc:260] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d73232888454dd48104ce7952fcf1d9" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 40651 } } peers { permanent_uuid: "328bfefe32fe490ea90b685743e0e5a3" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42009 } } peers { permanent_uuid: "fa16b73aac334b9cb7bbf92182c80872" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42903 } }
I20260809 19:29:27.991844 8515 consensus_queue.cc:260] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d73232888454dd48104ce7952fcf1d9" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 40651 } } peers { permanent_uuid: "328bfefe32fe490ea90b685743e0e5a3" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42009 } } peers { permanent_uuid: "fa16b73aac334b9cb7bbf92182c80872" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42903 } }
I20260809 19:29:27.991981 8514 ts_tablet_manager.cc:1428] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:29:27.992108 8363 heartbeater.cc:503] Master 127.21.81.254:35963 was elected leader, sending a full tablet report...
I20260809 19:29:27.992167 8515 ts_tablet_manager.cc:1428] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:29:27.992301 8499 heartbeater.cc:503] Master 127.21.81.254:35963 was elected leader, sending a full tablet report...
I20260809 19:29:27.993526 8516 raft_consensus.cc:348] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d73232888454dd48104ce7952fcf1d9" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 40651 } } peers { permanent_uuid: "328bfefe32fe490ea90b685743e0e5a3" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42009 } } peers { permanent_uuid: "fa16b73aac334b9cb7bbf92182c80872" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42903 } }
I20260809 19:29:27.993705 8516 raft_consensus.cc:374] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:27.993729 8516 raft_consensus.cc:729] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 328bfefe32fe490ea90b685743e0e5a3, State: Initialized, Role: FOLLOWER
I20260809 19:29:27.993825 8516 consensus_queue.cc:260] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d73232888454dd48104ce7952fcf1d9" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 40651 } } peers { permanent_uuid: "328bfefe32fe490ea90b685743e0e5a3" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42009 } } peers { permanent_uuid: "fa16b73aac334b9cb7bbf92182c80872" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42903 } }
I20260809 19:29:27.994033 8516 ts_tablet_manager.cc:1428] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 19:29:27.994112 8226 heartbeater.cc:503] Master 127.21.81.254:35963 was elected leader, sending a full tablet report...
W20260809 19:29:28.083658 8364 tablet.cc:2369] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:29:28.157473 8523 raft_consensus.cc:482] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:29:28.157718 8523 raft_consensus.cc:504] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d73232888454dd48104ce7952fcf1d9" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 40651 } } peers { permanent_uuid: "328bfefe32fe490ea90b685743e0e5a3" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42009 } } peers { permanent_uuid: "fa16b73aac334b9cb7bbf92182c80872" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42903 } }
I20260809 19:29:28.158169 8523 leader_election.cc:288] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d73232888454dd48104ce7952fcf1d9 (127.21.81.194:40651), fa16b73aac334b9cb7bbf92182c80872 (127.21.81.195:42903)
I20260809 19:29:28.161222 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:28.158388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:42903 (local address 127.21.81.193:39021)
0809 19:29:28.158452 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:28.158466 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:28.158523 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:28.158720 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:28.158721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:28.158723 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:28.158814 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:28.158815 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:28.159472 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:28.159473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:28.160284 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:28.160287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:28.161015 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:28.161016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:28.161075 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:28.161078 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:29:28.161134 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 19:29:28.161166 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 19:29:28.161303 8509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:28.158531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:39021 (local address 127.21.81.195:42903)
0809 19:29:28.158588 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:28.158590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:28.158599 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:29:28.158637 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:28.158638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:28.158645 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:28.158714 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:28.158885 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:28.159379 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:28.160366 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:28.160933 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:28.161010 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:28.161019 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:28.161162 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 19:29:28.161202 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:29:28.161635 8524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:28.158408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:40651 (local address 127.21.81.193:35357)
0809 19:29:28.158698 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:28.158707 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:28.158792 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:28.158950 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:28.158950 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:28.158952 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:28.159029 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:28.159031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:28.159690 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:28.159691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:28.160562 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:28.160565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:28.161327 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:28.161328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:28.161407 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:28.161411 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:29:28.161498 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 19:29:28.161533 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":80,"thread_start_us":140,"threads_started":1}
I20260809 19:29:28.161597 8452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "311d057fd3494c99b2a02028a28a2bd0" candidate_uuid: "328bfefe32fe490ea90b685743e0e5a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa16b73aac334b9cb7bbf92182c80872" is_pre_election: true
I20260809 19:29:28.161651 8511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:28.158455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:35357 (local address 127.21.81.194:40651)
0809 19:29:28.158527 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:28.158530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:28.158782 (+ 252us) server_negotiation.cc:407] Connection header received
0809 19:29:28.158853 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:28.158854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:28.158868 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:28.158945 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:28.159104 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:28.159577 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:28.160657 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:28.161214 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:28.161293 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:28.161301 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:28.161515 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 19:29:28.161562 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 19:29:28.161757 8452 raft_consensus.cc:2454] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 328bfefe32fe490ea90b685743e0e5a3 in term 0.
I20260809 19:29:28.161947 8314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "311d057fd3494c99b2a02028a28a2bd0" candidate_uuid: "328bfefe32fe490ea90b685743e0e5a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d73232888454dd48104ce7952fcf1d9" is_pre_election: true
I20260809 19:29:28.161984 8108 leader_election.cc:302] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 328bfefe32fe490ea90b685743e0e5a3, fa16b73aac334b9cb7bbf92182c80872; no voters:
I20260809 19:29:28.162111 8314 raft_consensus.cc:2454] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 328bfefe32fe490ea90b685743e0e5a3 in term 0.
I20260809 19:29:28.162137 8523 raft_consensus.cc:2781] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:29:28.162196 8523 raft_consensus.cc:482] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:29:28.162344 8523 raft_consensus.cc:3037] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:28.163082 8523 raft_consensus.cc:504] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d73232888454dd48104ce7952fcf1d9" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 40651 } } peers { permanent_uuid: "328bfefe32fe490ea90b685743e0e5a3" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42009 } } peers { permanent_uuid: "fa16b73aac334b9cb7bbf92182c80872" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42903 } }
I20260809 19:29:28.163235 8523 leader_election.cc:288] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [CANDIDATE]: Term 1 election: Requested vote from peers 6d73232888454dd48104ce7952fcf1d9 (127.21.81.194:40651), fa16b73aac334b9cb7bbf92182c80872 (127.21.81.195:42903)
I20260809 19:29:28.163381 8314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "311d057fd3494c99b2a02028a28a2bd0" candidate_uuid: "328bfefe32fe490ea90b685743e0e5a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d73232888454dd48104ce7952fcf1d9"
I20260809 19:29:28.163422 8452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "311d057fd3494c99b2a02028a28a2bd0" candidate_uuid: "328bfefe32fe490ea90b685743e0e5a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa16b73aac334b9cb7bbf92182c80872"
I20260809 19:29:28.163475 8314 raft_consensus.cc:3037] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:28.163522 8452 raft_consensus.cc:3037] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:28.164261 8314 raft_consensus.cc:2454] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 328bfefe32fe490ea90b685743e0e5a3 in term 1.
I20260809 19:29:28.164340 8452 raft_consensus.cc:2454] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 328bfefe32fe490ea90b685743e0e5a3 in term 1.
I20260809 19:29:28.164521 8108 leader_election.cc:302] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 328bfefe32fe490ea90b685743e0e5a3, fa16b73aac334b9cb7bbf92182c80872; no voters:
I20260809 19:29:28.164644 8523 raft_consensus.cc:2781] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:28.164855 8523 raft_consensus.cc:686] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [term 1 LEADER]: Becoming Leader. State: Replica: 328bfefe32fe490ea90b685743e0e5a3, State: Running, Role: LEADER
I20260809 19:29:28.164964 8523 consensus_queue.cc:237] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d73232888454dd48104ce7952fcf1d9" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 40651 } } peers { permanent_uuid: "328bfefe32fe490ea90b685743e0e5a3" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42009 } } peers { permanent_uuid: "fa16b73aac334b9cb7bbf92182c80872" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42903 } }
I20260809 19:29:28.165918 8028 catalog_manager.cc:5495] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 328bfefe32fe490ea90b685743e0e5a3 (127.21.81.193). New cstate: current_term: 1 leader_uuid: "328bfefe32fe490ea90b685743e0e5a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d73232888454dd48104ce7952fcf1d9" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 40651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "328bfefe32fe490ea90b685743e0e5a3" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 42009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa16b73aac334b9cb7bbf92182c80872" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42903 } health_report { overall_health: UNKNOWN } } }
W20260809 19:29:28.208698 8500 tablet.cc:2369] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:29:28.210191 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:28.207763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39554 (local address 127.21.81.193:42009)
0809 19:29:28.207836 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:28.207838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:28.207847 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:29:28.207906 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:28.207907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:28.207928 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:28.207986 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:29:28.208118 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:28.208631 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:28.209416 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:28.209638 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:28.209693 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:28.209835 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:29:28.209931 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:29:28.209990 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:28.210026 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 19:29:28.210056 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 19:29:28.211997 8452 raft_consensus.cc:1261] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872 [term 1 FOLLOWER]: Refusing update from remote peer 328bfefe32fe490ea90b685743e0e5a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:29:28.212000 8314 raft_consensus.cc:1261] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9 [term 1 FOLLOWER]: Refusing update from remote peer 328bfefe32fe490ea90b685743e0e5a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:29:28.212369 8525 consensus_queue.cc:1035] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d73232888454dd48104ce7952fcf1d9" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 40651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:29:28.212528 8523 consensus_queue.cc:1035] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa16b73aac334b9cb7bbf92182c80872" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 19:29:28.212787 8227 tablet.cc:2369] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:29:28.215960 8530 mvcc.cc:205] Tried to move back new op lower bound from 7316700234593656832 to 7316700234404261888. Current Snapshot: MvccSnapshot[applied={T|T < 7316700234593656832 or (T in {7316700234593656832})}]
I20260809 19:29:28.216080 8529 mvcc.cc:205] Tried to move back new op lower bound from 7316700234593656832 to 7316700234404261888. Current Snapshot: MvccSnapshot[applied={T|T < 7316700234593656832 or (T in {7316700234593656832})}]
I20260809 19:29:28.217022 8029 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:29:28.217114 8029 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:29:28.217865 8029 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7f2f364bc5f049d9a29ba888e40467a0: Sending DeleteTablet for 3 replicas of tablet 311d057fd3494c99b2a02028a28a2bd0
I20260809 19:29:28.218191 8429 tablet_service.cc:1514] Processing DeleteTablet for tablet 311d057fd3494c99b2a02028a28a2bd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:28 UTC) from {username='slave'} at 127.0.0.1:49336
I20260809 19:29:28.218191 8294 tablet_service.cc:1514] Processing DeleteTablet for tablet 311d057fd3494c99b2a02028a28a2bd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:28 UTC) from {username='slave'} at 127.0.0.1:48338
I20260809 19:29:28.218393 8157 tablet_service.cc:1514] Processing DeleteTablet for tablet 311d057fd3494c99b2a02028a28a2bd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:28 UTC) from {username='slave'} at 127.0.0.1:39540
I20260809 19:29:28.218637 8540 tablet_replica.cc:330] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3: stopping tablet replica
I20260809 19:29:28.218794 8540 raft_consensus.cc:2229] T 311d057fd3494c99b2a02028a28a2bd0 P 328bfefe32fe490ea90b685743e0e5a3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:29:28.218894 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 8092
I20260809 19:29:28.222649 8539 tablet_replica.cc:330] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9: stopping tablet replica
I20260809 19:29:28.222738 8539 raft_consensus.cc:2229] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:29:28.222858 8539 raft_consensus.cc:2258] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:29:28.223054 8538 tablet_replica.cc:330] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872: stopping tablet replica
I20260809 19:29:28.223153 8538 raft_consensus.cc:2229] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:29:28.223258 8538 raft_consensus.cc:2258] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:29:28.223333 8539 ts_tablet_manager.cc:1905] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:29:28.223694 8538 ts_tablet_manager.cc:1905] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:29:28.225231 8539 ts_tablet_manager.cc:1918] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:29:28.225311 8539 log.cc:1196] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/311d057fd3494c99b2a02028a28a2bd0
I20260809 19:29:28.225443 8538 ts_tablet_manager.cc:1918] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:29:28.225518 8538 log.cc:1196] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/311d057fd3494c99b2a02028a28a2bd0
I20260809 19:29:28.225672 8539 ts_tablet_manager.cc:1939] T 311d057fd3494c99b2a02028a28a2bd0 P 6d73232888454dd48104ce7952fcf1d9: Deleting consensus metadata
I20260809 19:29:28.225844 8538 ts_tablet_manager.cc:1939] T 311d057fd3494c99b2a02028a28a2bd0 P fa16b73aac334b9cb7bbf92182c80872: Deleting consensus metadata
I20260809 19:29:28.226154 8014 catalog_manager.cc:4841] TS 6d73232888454dd48104ce7952fcf1d9 (127.21.81.194:40651): tablet 311d057fd3494c99b2a02028a28a2bd0 (table test-table [id=93b04ac790c248e5a2e7921975149ea4]) successfully deleted
I20260809 19:29:28.226377 8012 catalog_manager.cc:4841] TS fa16b73aac334b9cb7bbf92182c80872 (127.21.81.195:42903): tablet 311d057fd3494c99b2a02028a28a2bd0 (table test-table [id=93b04ac790c248e5a2e7921975149ea4]) successfully deleted
W20260809 19:29:28.227576 8013 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:42009 (error 108)
I20260809 19:29:28.227787 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 8229
I20260809 19:29:28.228586 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:28.227914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:42009 (local address 127.0.0.1:39566)
0809 19:29:28.228502 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:28.228548 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:42009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":564}
W20260809 19:29:28.228967 8013 catalog_manager.cc:4584] TS 328bfefe32fe490ea90b685743e0e5a3 (127.21.81.193:42009): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 311d057fd3494c99b2a02028a28a2bd0: Network error: Client connection negotiation failed: client connection to 127.21.81.193:42009: connect: Connection refused (error 111)
I20260809 19:29:28.236076 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 8366
I20260809 19:29:28.242658 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 7995
2026-08-09T19:29:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (853 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:28 dist-test-slave-njxd krb5kdc[8544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:28 dist-test-slave-njxd krb5kdc[8544](info): set up 2 sockets
Aug 09 19:29:28 dist-test-slave-njxd krb5kdc[8544](info): commencing operation
krb5kdc: starting...
W20260809 19:29:29.316620 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:29 dist-test-slave-njxd krb5kdc[8544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:29:29.439821 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:40907
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:34225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:40907
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:29.520740 8559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:29.520929 8559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:29.520951 8559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:29.522020 8559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:29.522053 8559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:29.522068 8559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:29.522079 8559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:29.522090 8559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:29.523455 8559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:34225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:40907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:40907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:29.523628 8559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:29.523787 8559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:29.528481 8564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:29.528484 8567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:29.528510 8565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:29.528687 8559 server_base.cc:1034] running on GCE node
I20260809 19:29:29.528961 8559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:29.529325 8559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:29.530462 8559 hybrid_clock.cc:648] HybridClock initialized: now 1786303769530443 us; error 39 us; skew 500 ppm
Aug 09 19:29:29 dist-test-slave-njxd krb5kdc[8544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303769, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:29.532984 8559 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:29:29.533422 8559 webserver.cc:460] Webserver started at http://127.21.81.254:46153/ using document root <none> and password file <none>
I20260809 19:29:29.533691 8559 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:29.533737 8559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:29.533875 8559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:29.534881 8559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "59307095d3bf4c8fb149737024fb86bf"
format_stamp: "Formatted at 2026-08-09 19:29:29 on dist-test-slave-njxd"
server_key: "f92f3427e610bf8e9605ee6d08bd06c3"
server_key_iv: "2608400685cfe419f60624cdadb757c3"
server_key_version: "encryptionkey@0"
I20260809 19:29:29.535233 8559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "59307095d3bf4c8fb149737024fb86bf"
format_stamp: "Formatted at 2026-08-09 19:29:29 on dist-test-slave-njxd"
server_key: "f92f3427e610bf8e9605ee6d08bd06c3"
server_key_iv: "2608400685cfe419f60624cdadb757c3"
server_key_version: "encryptionkey@0"
I20260809 19:29:29.536582 8559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:29:29.537310 8575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:29.537493 8559 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:29:29.537628 8559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "59307095d3bf4c8fb149737024fb86bf"
format_stamp: "Formatted at 2026-08-09 19:29:29 on dist-test-slave-njxd"
server_key: "f92f3427e610bf8e9605ee6d08bd06c3"
server_key_iv: "2608400685cfe419f60624cdadb757c3"
server_key_version: "encryptionkey@0"
I20260809 19:29:29.537701 8559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:29.557834 8559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:29.559675 8559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:29.559821 8559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:29.563973 8559 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:40907
I20260809 19:29:29.564060 8631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:40907 every 8 connection(s)
I20260809 19:29:29.564437 8559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:29.564895 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 8559
I20260809 19:29:29.565003 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:29.565210 21831 external_mini_cluster.cc:1428] Setting key d3051e0dcc3a95a4bc2fc44722972ce9
I20260809 19:29:29.565518 8632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:29.568899 8632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf: Bootstrap starting.
I20260809 19:29:29.569679 8632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:29.570015 8632 log.cc:824] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:29.570786 8632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf: No bootstrap required, opened a new log
Aug 09 19:29:29 dist-test-slave-njxd krb5kdc[8544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:29:29.573213 8632 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59307095d3bf4c8fb149737024fb86bf" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40907 } }
I20260809 19:29:29.573343 8632 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:29.573365 8632 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59307095d3bf4c8fb149737024fb86bf, State: Initialized, Role: FOLLOWER
I20260809 19:29:29.573607 8632 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59307095d3bf4c8fb149737024fb86bf" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40907 } }
I20260809 19:29:29.573724 8632 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:29.573761 8632 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:29.573818 8632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:29.574712 8632 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59307095d3bf4c8fb149737024fb86bf" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40907 } }
I20260809 19:29:29.574827 8632 leader_election.cc:302] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59307095d3bf4c8fb149737024fb86bf; no voters:
I20260809 19:29:29.574990 8632 leader_election.cc:288] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:29.575055 8637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:29.575249 8632 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:29.575274 8637 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [term 1 LEADER]: Becoming Leader. State: Replica: 59307095d3bf4c8fb149737024fb86bf, State: Running, Role: LEADER
I20260809 19:29:29.575497 8637 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59307095d3bf4c8fb149737024fb86bf" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40907 } }
I20260809 19:29:29.575874 8639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59307095d3bf4c8fb149737024fb86bf. Latest consensus state: current_term: 1 leader_uuid: "59307095d3bf4c8fb149737024fb86bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59307095d3bf4c8fb149737024fb86bf" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40907 } } }
I20260809 19:29:29.575955 8639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:29.576674 8638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59307095d3bf4c8fb149737024fb86bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59307095d3bf4c8fb149737024fb86bf" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 40907 } } }
I20260809 19:29:29.576767 8638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:29.576792 8642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:29.577062 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:29.565876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39678 (local address 127.21.81.254:40907)
0809 19:29:29.566051 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:29.566059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:29.566086 (+ 27us) server_negotiation.cc:407] Connection header received
0809 19:29:29.566693 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:29.566700 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:29.566749 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:29.566858 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:29.567703 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:29.568547 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:29.569662 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:29.570154 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:29.570280 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:29.573015 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:29.573018 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:29.573018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:29.573028 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:29.575143 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:29.575713 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:29.575716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:29.575716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:29.575841 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:29.576056 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:29.576057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:29.576058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:29.576364 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:29.576525 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:29.576652 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 19:29:29.576816 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260809 19:29:29.577396 8642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:29.579028 8642 catalog_manager.cc:1347] Generated new cluster ID: 390c98e745274962ab15278780d84440
I20260809 19:29:29.579088 8642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:29.590826 8642 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:29.591543 8642 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:29.600054 8642 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf: Generated new TSK 0
I20260809 19:29:29.600256 8642 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:29:29.636593 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:40907
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:34225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:29.720942 8659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:29.721218 8659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:29.721263 8659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:29.722473 8659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:29.722553 8659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:29.722610 8659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:29:29.724130 8659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:34225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:40907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:29.724404 8659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:29.724664 8659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:29:29.729944 8659 server_base.cc:1034] running on GCE node
W20260809 19:29:29.729923 8665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:29.729923 8667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:29.729908 8664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:29.730371 8659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:29.730641 8659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:29.731810 8659 hybrid_clock.cc:648] HybridClock initialized: now 1786303769731806 us; error 37 us; skew 500 ppm
Aug 09 19:29:29 dist-test-slave-njxd krb5kdc[8544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303769, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:29.734536 8659 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:29:29.735002 8659 webserver.cc:460] Webserver started at http://127.21.81.193:36317/ using document root <none> and password file <none>
I20260809 19:29:29.735242 8659 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:29.735314 8659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:29.735466 8659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:29.736539 8659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "523a9a06368e4af98323740cf9a15f43"
format_stamp: "Formatted at 2026-08-09 19:29:29 on dist-test-slave-njxd"
server_key: "c3eb6ff6833a4b18c2a924701388684e"
server_key_iv: "609cdff060698715f843df88be265fbe"
server_key_version: "encryptionkey@0"
I20260809 19:29:29.736928 8659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "523a9a06368e4af98323740cf9a15f43"
format_stamp: "Formatted at 2026-08-09 19:29:29 on dist-test-slave-njxd"
server_key: "c3eb6ff6833a4b18c2a924701388684e"
server_key_iv: "609cdff060698715f843df88be265fbe"
server_key_version: "encryptionkey@0"
I20260809 19:29:29.738376 8659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:29:29.739135 8674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:29.739313 8659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:29.739423 8659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "523a9a06368e4af98323740cf9a15f43"
format_stamp: "Formatted at 2026-08-09 19:29:29 on dist-test-slave-njxd"
server_key: "c3eb6ff6833a4b18c2a924701388684e"
server_key_iv: "609cdff060698715f843df88be265fbe"
server_key_version: "encryptionkey@0"
I20260809 19:29:29.739498 8659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:29.751235 8659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:29.753551 8659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:29.753767 8659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:29.754029 8659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:29.754343 8659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:29.754397 8659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:29.754442 8659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:29.754475 8659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:29.760334 8659 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:44823
I20260809 19:29:29.760389 8794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:44823 every 8 connection(s)
I20260809 19:29:29.760720 8659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:29:29.762532 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 8659
I20260809 19:29:29.762672 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:29:29.762889 21831 external_mini_cluster.cc:1428] Setting key e9c145dca9106132e8830e5a39a24264
Aug 09 19:29:29 dist-test-slave-njxd krb5kdc[8544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303769, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:29:29.770884 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:29.761401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:45235 (local address 127.21.81.254:40907)
0809 19:29:29.761454 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:29.761456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:29.761678 (+ 222us) server_negotiation.cc:407] Connection header received
0809 19:29:29.762185 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:29.762185 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:29.762195 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:29.762258 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:29.763447 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:29.764027 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:29.765096 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:29.765461 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:29.765537 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:29.767854 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:29.767856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:29.767856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:29.767864 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:29.769285 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:29.769811 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:29.769812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:29.769813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:29.769878 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:29.770159 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:29.770160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:29.770160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:29.770285 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:29.770360 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:29.770680 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 19:29:29.770781 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:29:29.771155 8797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:29.761368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:40907 (local address 127.21.81.193:45235)
0809 19:29:29.761533 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:29.761599 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:29.762099 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:29.762340 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:29.762343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:29.763079 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:29.763343 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:29.763350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:29.764134 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:29.764135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:29.764987 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:29.764990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:29.765557 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:29.765558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:29.765745 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:29.766273 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:29.766281 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:29.767757 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:29.769380 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:29.769389 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:29.769391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:29.769717 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:29.769949 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:29.769950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:29.769951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:29.770074 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:29.770363 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:29.770371 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:29.770519 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 19:29:29.770624 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 19:29:29.770914 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260809 19:29:29.771714 8795 heartbeater.cc:347] Connected to a master server at 127.21.81.254:40907
I20260809 19:29:29.771813 8795 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:29.772022 8795 heartbeater.cc:511] Master 127.21.81.254:40907 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
I20260809 19:29:29.772593 8593 ts_manager.cc:194] Registered new tserver with Master: 523a9a06368e4af98323740cf9a15f43 (127.21.81.193:44823)
I20260809 19:29:29.773309 8593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:45235
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:29:29.791923 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:40907
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:34225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:29.875560 8800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:29.875816 8800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:29.875855 8800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:29.877198 8800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:29.877245 8800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:29.877272 8800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:29:29.878775 8800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:34225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:40907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:29.878980 8800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:29.879159 8800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:29:29.884214 8800 server_base.cc:1034] running on GCE node
W20260809 19:29:29.884186 8808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:29.884199 8806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:29.884199 8805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:29.884619 8800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:29.884871 8800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:29.886014 8800 hybrid_clock.cc:648] HybridClock initialized: now 1786303769886001 us; error 54 us; skew 500 ppm
Aug 09 19:29:29 dist-test-slave-njxd krb5kdc[8544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303769, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:29.888504 8800 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:29:29.888933 8800 webserver.cc:460] Webserver started at http://127.21.81.194:42351/ using document root <none> and password file <none>
I20260809 19:29:29.889156 8800 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:29.889225 8800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:29.889379 8800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:29.890457 8800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "05e974187cfd441495254c357dc4ee90"
format_stamp: "Formatted at 2026-08-09 19:29:29 on dist-test-slave-njxd"
server_key: "1d718f6752bb0ac020bf86741af51207"
server_key_iv: "a5199f86e270d2cf90743d94297842d5"
server_key_version: "encryptionkey@0"
I20260809 19:29:29.890818 8800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "05e974187cfd441495254c357dc4ee90"
format_stamp: "Formatted at 2026-08-09 19:29:29 on dist-test-slave-njxd"
server_key: "1d718f6752bb0ac020bf86741af51207"
server_key_iv: "a5199f86e270d2cf90743d94297842d5"
server_key_version: "encryptionkey@0"
I20260809 19:29:29.892155 8800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:29:29.892882 8816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:29.893055 8800 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:29.893153 8800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "05e974187cfd441495254c357dc4ee90"
format_stamp: "Formatted at 2026-08-09 19:29:29 on dist-test-slave-njxd"
server_key: "1d718f6752bb0ac020bf86741af51207"
server_key_iv: "a5199f86e270d2cf90743d94297842d5"
server_key_version: "encryptionkey@0"
I20260809 19:29:29.893236 8800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:29.914264 8800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:29.916556 8800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:29.916728 8800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:29.916962 8800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:29.917325 8800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:29.917377 8800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:29.917413 8800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:29.917454 8800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:29.923379 8800 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:33763
I20260809 19:29:29.923434 8938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:33763 every 8 connection(s)
I20260809 19:29:29.923719 8800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:29:29.927480 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 8800
I20260809 19:29:29.927574 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:29:29.927790 21831 external_mini_cluster.cc:1428] Setting key 375ba54d789120ea0a95ac5e30df382d
Aug 09 19:29:29 dist-test-slave-njxd krb5kdc[8544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303769, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:29:29.933893 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:29.924389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:47933 (local address 127.21.81.254:40907)
0809 19:29:29.924456 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:29.924458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:29.924626 (+ 168us) server_negotiation.cc:407] Connection header received
0809 19:29:29.925108 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:29.925109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:29.925123 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:29.925200 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:29.926229 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:29.926761 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:29.927940 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:29.928236 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:29.928356 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:29.930801 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:29.930802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:29.930803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:29.930809 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:29.932408 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:29.932827 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:29.932829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:29.932829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:29.932895 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:29.933140 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:29.933141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:29.933142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:29.933304 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:29.933382 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:29.933690 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 19:29:29.933787 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:29:29.934036 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:29.924340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:40907 (local address 127.21.81.194:47933)
0809 19:29:29.924501 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:29.924541 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:29.925015 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:29.925271 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:29.925292 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:29.925921 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:29.926129 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:29.926135 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:29.926885 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:29.926887 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:29.927805 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:29.927807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:29.928367 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:29.928368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:29.928495 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:29.928971 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:29.928981 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:29.930679 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:29.932498 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:29.932505 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:29.932506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:29.932747 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:29.932965 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:29.932966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:29.932966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:29.933067 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:29.933376 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:29.933381 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:29.933493 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 19:29:29.933656 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 19:29:29.933785 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260809 19:29:29.934706 8939 heartbeater.cc:347] Connected to a master server at 127.21.81.254:40907
I20260809 19:29:29.934818 8939 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:29.935031 8939 heartbeater.cc:511] Master 127.21.81.254:40907 requested a full tablet report, sending...
I20260809 19:29:29.935431 8593 ts_manager.cc:194] Registered new tserver with Master: 05e974187cfd441495254c357dc4ee90 (127.21.81.194:33763)
I20260809 19:29:29.935834 8593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:47933
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:29:29.954501 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:40907
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:34225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:30.037861 8944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:30.038059 8944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:30.038084 8944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:30.039296 8944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:30.039335 8944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:30.039361 8944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:29:30.040805 8944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:34225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:40907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:30.041025 8944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:30.041210 8944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:30.046427 8949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:30.046397 8950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:30.046545 8952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:30.046806 8944 server_base.cc:1034] running on GCE node
I20260809 19:29:30.047021 8944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:30.047289 8944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:30.048462 8944 hybrid_clock.cc:648] HybridClock initialized: now 1786303770048457 us; error 38 us; skew 500 ppm
Aug 09 19:29:30 dist-test-slave-njxd krb5kdc[8544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303770, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:30.050998 8944 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:29:30.051499 8944 webserver.cc:460] Webserver started at http://127.21.81.195:44947/ using document root <none> and password file <none>
I20260809 19:29:30.051721 8944 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:30.051787 8944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:30.051934 8944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:30.052978 8944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "d9d10408cba64af0b4c2c8085cb668e8"
format_stamp: "Formatted at 2026-08-09 19:29:30 on dist-test-slave-njxd"
server_key: "aac939477cee83f51837fc4e45d5bc3f"
server_key_iv: "db2d93ebe601aaaee9ab9ca59878be80"
server_key_version: "encryptionkey@0"
I20260809 19:29:30.053354 8944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "d9d10408cba64af0b4c2c8085cb668e8"
format_stamp: "Formatted at 2026-08-09 19:29:30 on dist-test-slave-njxd"
server_key: "aac939477cee83f51837fc4e45d5bc3f"
server_key_iv: "db2d93ebe601aaaee9ab9ca59878be80"
server_key_version: "encryptionkey@0"
I20260809 19:29:30.054754 8944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:29:30.055472 8959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:30.055656 8944 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:30.055753 8944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "d9d10408cba64af0b4c2c8085cb668e8"
format_stamp: "Formatted at 2026-08-09 19:29:30 on dist-test-slave-njxd"
server_key: "aac939477cee83f51837fc4e45d5bc3f"
server_key_iv: "db2d93ebe601aaaee9ab9ca59878be80"
server_key_version: "encryptionkey@0"
I20260809 19:29:30.055835 8944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:30.082957 8944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:30.084981 8944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:30.085147 8944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:30.085402 8944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:30.085772 8944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:30.085826 8944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:30.085880 8944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:30.085911 8944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:30.091938 8944 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:37171
I20260809 19:29:30.091984 9080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:37171 every 8 connection(s)
I20260809 19:29:30.092293 8944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:29:30 dist-test-slave-njxd krb5kdc[8544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303770, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:29:30.099915 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 8944
I20260809 19:29:30.100065 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:29:30.100270 21831 external_mini_cluster.cc:1428] Setting key 80e3136d56c4a9df321dd6646fff9615
I20260809 19:29:30.101540 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.092961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:51253 (local address 127.21.81.254:40907)
0809 19:29:30.093004 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.093006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.093205 (+ 199us) server_negotiation.cc:407] Connection header received
0809 19:29:30.093788 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.093789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.093799 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.093843 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:30.094780 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.095269 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.096325 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.096502 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.096574 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.098720 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:30.098721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:30.098722 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:30.098728 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:30.099934 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:30.100436 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:30.100437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:30.100438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:30.100529 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:30.100822 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:30.100823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:30.100823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:30.100983 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:30.101044 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.101361 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.101449 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:29:30.101776 9083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.092919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:40907 (local address 127.21.81.195:51253)
0809 19:29:30.093083 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:30.093124 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:30.093683 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:30.093914 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:30.093917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:30.094475 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:30.094704 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.094709 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.095349 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.095349 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.096256 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.096258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.096574 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.096575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.096667 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.097225 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:30.097238 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:30.098634 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:30.100014 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:30.100023 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:30.100025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:30.100349 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:30.100587 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:30.100589 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:30.100589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:30.100766 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:30.101047 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:30.101055 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:30.101215 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 19:29:30.101337 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 19:29:30.101499 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260809 19:29:30.102205 9081 heartbeater.cc:347] Connected to a master server at 127.21.81.254:40907
I20260809 19:29:30.102306 9081 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:30.102461 9081 heartbeater.cc:511] Master 127.21.81.254:40907 requested a full tablet report, sending...
I20260809 19:29:30.102774 8593 ts_manager.cc:194] Registered new tserver with Master: d9d10408cba64af0b4c2c8085cb668e8 (127.21.81.195:37171)
I20260809 19:29:30.103093 8593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:51253
I20260809 19:29:30.112221 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:29:30.119401 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.113540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39688 (local address 127.21.81.254:40907)
0809 19:29:30.113614 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.113616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.113746 (+ 130us) server_negotiation.cc:407] Connection header received
0809 19:29:30.113794 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.113794 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.113803 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.113851 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:30.114376 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.114859 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.115522 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.115694 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.115740 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.117028 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:30.117029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:30.117029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:30.117034 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:30.118352 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:30.118702 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:30.118702 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:30.118703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:30.118752 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:30.118942 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:30.118942 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:30.118942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:30.119058 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:30.119164 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.119246 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.119293 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 19:29:30.121158 8593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:29:30.122931 8593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:29:30.129908 9094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.125509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49324 (local address 127.21.81.195:37171)
0809 19:29:30.125899 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.125902 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.125912 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:29:30.125955 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.125957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.125986 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.126081 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:30.126273 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.127398 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.128739 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.129442 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.129522 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.129608 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.129782 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.129826 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":129,"threads_started":1}
I20260809 19:29:30.129899 9092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.125383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:37171 (local address 127.0.0.1:49324)
0809 19:29:30.125676 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:30.125718 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:30.125789 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:30.126093 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:30.126096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:30.126098 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:30.126182 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.126184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.127515 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.127516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.128643 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.128646 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.129527 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.129528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.129652 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.129665 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:29:30.129751 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 19:29:30.129804 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":43,"thread_start_us":69,"threads_started":1}
I20260809 19:29:30.130095 9095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.125596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40028 (local address 127.21.81.193:44823)
0809 19:29:30.125810 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.125815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.126195 (+ 380us) server_negotiation.cc:407] Connection header received
0809 19:29:30.126256 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.126259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.126395 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.126480 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:30.126631 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.127676 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.128949 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.129673 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.129757 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.129815 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.129954 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.130009 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":93,"threads_started":1}
I20260809 19:29:30.130300 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.125397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:44823 (local address 127.0.0.1:40028)
0809 19:29:30.126062 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:30.126072 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:30.126126 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:30.126501 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:30.126502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:30.126504 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:30.126552 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.126553 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.127777 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.127777 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.128861 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.128863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.129747 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.129748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.129803 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.129806 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:29:30.129862 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 19:29:30.129893 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":78,"threads_started":1}
I20260809 19:29:30.131029 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.125929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:33763 (local address 127.0.0.1:56964)
0809 19:29:30.126211 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:30.126221 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:30.126273 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:30.126601 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:30.126601 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:30.126603 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:30.126657 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.126658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.127529 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.127530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.128856 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.128859 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.130794 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.130795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.130866 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.130871 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:29:30.130921 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 19:29:30.130961 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":65,"thread_start_us":81,"threads_started":1}
I20260809 19:29:30.131049 8727 tablet_service.cc:1467] Processing CreateTablet for tablet 1ca60f78c34849fa8ccd4e82ef746f61 (DEFAULT_TABLE table=test-table [id=79ac2ce753ac4dd0bebf1ca4f173ffbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:30.131138 9011 tablet_service.cc:1467] Processing CreateTablet for tablet 1ca60f78c34849fa8ccd4e82ef746f61 (DEFAULT_TABLE table=test-table [id=79ac2ce753ac4dd0bebf1ca4f173ffbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:30.131426 9096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.125744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56964 (local address 127.21.81.194:33763)
0809 19:29:30.125922 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.125926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.126344 (+ 418us) server_negotiation.cc:407] Connection header received
0809 19:29:30.126396 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.126398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.126415 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.126518 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:30.126732 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.127421 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.129854 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.130618 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.130689 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.130757 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.131298 (+ 541us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.131348 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260809 19:29:30.131462 8727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ca60f78c34849fa8ccd4e82ef746f61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:30.131512 9011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ca60f78c34849fa8ccd4e82ef746f61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:30.132038 8868 tablet_service.cc:1467] Processing CreateTablet for tablet 1ca60f78c34849fa8ccd4e82ef746f61 (DEFAULT_TABLE table=test-table [id=79ac2ce753ac4dd0bebf1ca4f173ffbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:30.132354 8868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ca60f78c34849fa8ccd4e82ef746f61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:30.134574 9098 tablet_bootstrap.cc:492] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43: Bootstrap starting.
I20260809 19:29:30.134927 9099 tablet_bootstrap.cc:492] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8: Bootstrap starting.
I20260809 19:29:30.135329 9098 tablet_bootstrap.cc:654] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:30.135679 9099 tablet_bootstrap.cc:654] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:30.135666 9100 tablet_bootstrap.cc:492] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90: Bootstrap starting.
I20260809 19:29:30.136001 9098 log.cc:824] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:30.136305 9099 log.cc:824] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:30.136353 9100 tablet_bootstrap.cc:654] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:30.136880 9098 tablet_bootstrap.cc:492] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43: No bootstrap required, opened a new log
I20260809 19:29:30.136955 9100 log.cc:824] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:30.136962 9098 ts_tablet_manager.cc:1397] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:29:30.137658 9100 tablet_bootstrap.cc:492] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90: No bootstrap required, opened a new log
I20260809 19:29:30.137727 9100 ts_tablet_manager.cc:1397] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:29:30.138566 9098 raft_consensus.cc:348] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9d10408cba64af0b4c2c8085cb668e8" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37171 } } peers { permanent_uuid: "523a9a06368e4af98323740cf9a15f43" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44823 } } peers { permanent_uuid: "05e974187cfd441495254c357dc4ee90" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33763 } }
I20260809 19:29:30.138720 9098 raft_consensus.cc:374] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:30.138744 9098 raft_consensus.cc:729] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 523a9a06368e4af98323740cf9a15f43, State: Initialized, Role: FOLLOWER
I20260809 19:29:30.138870 9098 consensus_queue.cc:260] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9d10408cba64af0b4c2c8085cb668e8" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37171 } } peers { permanent_uuid: "523a9a06368e4af98323740cf9a15f43" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44823 } } peers { permanent_uuid: "05e974187cfd441495254c357dc4ee90" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33763 } }
I20260809 19:29:30.139166 9098 ts_tablet_manager.cc:1428] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:29:30.139236 8795 heartbeater.cc:503] Master 127.21.81.254:40907 was elected leader, sending a full tablet report...
I20260809 19:29:30.139330 9100 raft_consensus.cc:348] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9d10408cba64af0b4c2c8085cb668e8" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37171 } } peers { permanent_uuid: "523a9a06368e4af98323740cf9a15f43" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44823 } } peers { permanent_uuid: "05e974187cfd441495254c357dc4ee90" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33763 } }
I20260809 19:29:30.139456 9100 raft_consensus.cc:374] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:30.139545 9100 raft_consensus.cc:729] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05e974187cfd441495254c357dc4ee90, State: Initialized, Role: FOLLOWER
I20260809 19:29:30.139695 9100 consensus_queue.cc:260] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9d10408cba64af0b4c2c8085cb668e8" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37171 } } peers { permanent_uuid: "523a9a06368e4af98323740cf9a15f43" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44823 } } peers { permanent_uuid: "05e974187cfd441495254c357dc4ee90" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33763 } }
I20260809 19:29:30.139832 9099 tablet_bootstrap.cc:492] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8: No bootstrap required, opened a new log
I20260809 19:29:30.139904 9099 ts_tablet_manager.cc:1397] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 19:29:30.139976 9100 ts_tablet_manager.cc:1428] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:29:30.140149 8939 heartbeater.cc:503] Master 127.21.81.254:40907 was elected leader, sending a full tablet report...
I20260809 19:29:30.141903 9099 raft_consensus.cc:348] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9d10408cba64af0b4c2c8085cb668e8" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37171 } } peers { permanent_uuid: "523a9a06368e4af98323740cf9a15f43" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44823 } } peers { permanent_uuid: "05e974187cfd441495254c357dc4ee90" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33763 } }
I20260809 19:29:30.142033 9099 raft_consensus.cc:374] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:30.142055 9099 raft_consensus.cc:729] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9d10408cba64af0b4c2c8085cb668e8, State: Initialized, Role: FOLLOWER
I20260809 19:29:30.142184 9099 consensus_queue.cc:260] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9d10408cba64af0b4c2c8085cb668e8" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37171 } } peers { permanent_uuid: "523a9a06368e4af98323740cf9a15f43" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44823 } } peers { permanent_uuid: "05e974187cfd441495254c357dc4ee90" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33763 } }
I20260809 19:29:30.142450 9099 ts_tablet_manager.cc:1428] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:29:30.142513 9081 heartbeater.cc:503] Master 127.21.81.254:40907 was elected leader, sending a full tablet report...
W20260809 19:29:30.174247 8940 tablet.cc:2369] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:29:30.186157 9104 raft_consensus.cc:482] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:29:30.186282 9104 raft_consensus.cc:504] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9d10408cba64af0b4c2c8085cb668e8" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37171 } } peers { permanent_uuid: "523a9a06368e4af98323740cf9a15f43" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44823 } } peers { permanent_uuid: "05e974187cfd441495254c357dc4ee90" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33763 } }
I20260809 19:29:30.186765 9104 leader_election.cc:288] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9d10408cba64af0b4c2c8085cb668e8 (127.21.81.195:37171), 05e974187cfd441495254c357dc4ee90 (127.21.81.194:33763)
I20260809 19:29:30.189595 9094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.187004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:52155 (local address 127.21.81.195:37171)
0809 19:29:30.187066 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.187068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.187096 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:29:30.187124 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.187125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.187135 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.187206 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:30.187406 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.187886 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.188821 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.189213 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.189302 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.189311 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.189465 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.189508 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:29:30.189576 8797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.186925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:37171 (local address 127.21.81.193:52155)
0809 19:29:30.186974 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:30.186989 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:30.187038 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:30.187227 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:30.187239 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:30.187241 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:30.187313 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.187315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.187977 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.187978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.188721 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.188722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.189298 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.189299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.189385 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.189388 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:29:30.189457 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 19:29:30.189501 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":1}
I20260809 19:29:30.189797 9107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.186948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:33763 (local address 127.21.81.193:58735)
0809 19:29:30.187209 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:30.187217 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:30.187286 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:30.187445 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:30.187446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:30.187448 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:30.187529 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.187530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.188157 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.188158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.188893 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.188895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.189461 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.189462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.189544 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.189548 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:29:30.189668 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 19:29:30.189716 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":23,"thread_start_us":163,"threads_started":1}
I20260809 19:29:30.189844 9096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.187078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:58735 (local address 127.21.81.194:33763)
0809 19:29:30.187138 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.187140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.187281 (+ 141us) server_negotiation.cc:407] Connection header received
0809 19:29:30.187362 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.187369 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.187379 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.187442 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:30.187603 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.188059 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.188992 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.189372 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.189463 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.189471 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.189727 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.189772 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:29:30.189971 9033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ca60f78c34849fa8ccd4e82ef746f61" candidate_uuid: "523a9a06368e4af98323740cf9a15f43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9d10408cba64af0b4c2c8085cb668e8" is_pre_election: true
I20260809 19:29:30.190146 9033 raft_consensus.cc:2454] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 523a9a06368e4af98323740cf9a15f43 in term 0.
I20260809 19:29:30.190353 8889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ca60f78c34849fa8ccd4e82ef746f61" candidate_uuid: "523a9a06368e4af98323740cf9a15f43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05e974187cfd441495254c357dc4ee90" is_pre_election: true
I20260809 19:29:30.190436 8678 leader_election.cc:302] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 523a9a06368e4af98323740cf9a15f43, d9d10408cba64af0b4c2c8085cb668e8; no voters:
I20260809 19:29:30.190523 8889 raft_consensus.cc:2454] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 523a9a06368e4af98323740cf9a15f43 in term 0.
I20260809 19:29:30.190575 9104 raft_consensus.cc:2781] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:29:30.190685 9104 raft_consensus.cc:482] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:29:30.190730 9104 raft_consensus.cc:3037] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:30.191465 9104 raft_consensus.cc:504] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9d10408cba64af0b4c2c8085cb668e8" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37171 } } peers { permanent_uuid: "523a9a06368e4af98323740cf9a15f43" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44823 } } peers { permanent_uuid: "05e974187cfd441495254c357dc4ee90" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33763 } }
I20260809 19:29:30.191605 9104 leader_election.cc:288] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [CANDIDATE]: Term 1 election: Requested vote from peers d9d10408cba64af0b4c2c8085cb668e8 (127.21.81.195:37171), 05e974187cfd441495254c357dc4ee90 (127.21.81.194:33763)
I20260809 19:29:30.191807 9033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ca60f78c34849fa8ccd4e82ef746f61" candidate_uuid: "523a9a06368e4af98323740cf9a15f43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9d10408cba64af0b4c2c8085cb668e8"
I20260809 19:29:30.191905 9033 raft_consensus.cc:3037] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:30.192632 9033 raft_consensus.cc:2454] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 523a9a06368e4af98323740cf9a15f43 in term 1.
I20260809 19:29:30.192780 8889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ca60f78c34849fa8ccd4e82ef746f61" candidate_uuid: "523a9a06368e4af98323740cf9a15f43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05e974187cfd441495254c357dc4ee90"
I20260809 19:29:30.192828 8678 leader_election.cc:302] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 523a9a06368e4af98323740cf9a15f43, d9d10408cba64af0b4c2c8085cb668e8; no voters:
I20260809 19:29:30.192862 8889 raft_consensus.cc:3037] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:30.192903 9104 raft_consensus.cc:2781] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:30.193073 9104 raft_consensus.cc:686] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [term 1 LEADER]: Becoming Leader. State: Replica: 523a9a06368e4af98323740cf9a15f43, State: Running, Role: LEADER
I20260809 19:29:30.193173 9104 consensus_queue.cc:237] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9d10408cba64af0b4c2c8085cb668e8" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37171 } } peers { permanent_uuid: "523a9a06368e4af98323740cf9a15f43" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44823 } } peers { permanent_uuid: "05e974187cfd441495254c357dc4ee90" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33763 } }
I20260809 19:29:30.193878 8889 raft_consensus.cc:2454] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 523a9a06368e4af98323740cf9a15f43 in term 1.
I20260809 19:29:30.194160 8592 catalog_manager.cc:5495] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 reported cstate change: term changed from 0 to 1, leader changed from <none> to 523a9a06368e4af98323740cf9a15f43 (127.21.81.193). New cstate: current_term: 1 leader_uuid: "523a9a06368e4af98323740cf9a15f43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9d10408cba64af0b4c2c8085cb668e8" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "523a9a06368e4af98323740cf9a15f43" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 44823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05e974187cfd441495254c357dc4ee90" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33763 } health_report { overall_health: UNKNOWN } } }
I20260809 19:29:30.202489 9095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.200301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40034 (local address 127.21.81.193:44823)
0809 19:29:30.200344 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.200346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.200355 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:29:30.200431 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.200431 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.200446 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.200504 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:29:30.200610 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.201048 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.201806 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.201982 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.202034 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.202148 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:29:30.202224 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:29:30.202309 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.202353 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.202363 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:29:30.204249 9033 raft_consensus.cc:1261] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8 [term 1 FOLLOWER]: Refusing update from remote peer 523a9a06368e4af98323740cf9a15f43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:29:30.204253 8889 raft_consensus.cc:1261] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90 [term 1 FOLLOWER]: Refusing update from remote peer 523a9a06368e4af98323740cf9a15f43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:29:30.204599 9108 consensus_queue.cc:1035] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9d10408cba64af0b4c2c8085cb668e8" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:29:30.204731 9104 consensus_queue.cc:1035] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05e974187cfd441495254c357dc4ee90" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:29:30.207332 9112 mvcc.cc:205] Tried to move back new op lower bound from 7316700242753966080 to 7316700242711822336. Current Snapshot: MvccSnapshot[applied={T|T < 7316700242753966080}]
I20260809 19:29:30.207753 9113 mvcc.cc:205] Tried to move back new op lower bound from 7316700242753966080 to 7316700242711822336. Current Snapshot: MvccSnapshot[applied={T|T < 7316700242753966080}]
I20260809 19:29:30.209386 8592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:29:30.209486 8592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:29:30.210317 8592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59307095d3bf4c8fb149737024fb86bf: Sending DeleteTablet for 3 replicas of tablet 1ca60f78c34849fa8ccd4e82ef746f61
I20260809 19:29:30.210644 8727 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ca60f78c34849fa8ccd4e82ef746f61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:30 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:40028
I20260809 19:29:30.210822 9011 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ca60f78c34849fa8ccd4e82ef746f61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:30 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:49324
I20260809 19:29:30.210901 9121 tablet_replica.cc:330] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43: stopping tablet replica
I20260809 19:29:30.210894 8868 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ca60f78c34849fa8ccd4e82ef746f61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:30 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:56964
I20260809 19:29:30.211000 9121 raft_consensus.cc:2229] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:29:30.211159 9121 raft_consensus.cc:2258] T 1ca60f78c34849fa8ccd4e82ef746f61 P 523a9a06368e4af98323740cf9a15f43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:29:30.211257 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 8659
I20260809 19:29:30.212045 9123 tablet_replica.cc:330] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90: stopping tablet replica
I20260809 19:29:30.212159 9123 raft_consensus.cc:2229] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:29:30.212294 9123 raft_consensus.cc:2258] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:29:30.213068 9123 ts_tablet_manager.cc:1905] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:29:30.214485 9122 tablet_replica.cc:330] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8: stopping tablet replica
I20260809 19:29:30.214612 9122 raft_consensus.cc:2229] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:29:30.214771 9122 raft_consensus.cc:2258] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:29:30.215167 9123 ts_tablet_manager.cc:1918] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:29:30.215258 9123 log.cc:1196] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/1ca60f78c34849fa8ccd4e82ef746f61
I20260809 19:29:30.215245 9122 ts_tablet_manager.cc:1905] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:29:30.215593 9123 ts_tablet_manager.cc:1939] T 1ca60f78c34849fa8ccd4e82ef746f61 P 05e974187cfd441495254c357dc4ee90: Deleting consensus metadata
I20260809 19:29:30.216018 8578 catalog_manager.cc:4841] TS 05e974187cfd441495254c357dc4ee90 (127.21.81.194:33763): tablet 1ca60f78c34849fa8ccd4e82ef746f61 (table test-table [id=79ac2ce753ac4dd0bebf1ca4f173ffbe]) successfully deleted
I20260809 19:29:30.216709 9122 ts_tablet_manager.cc:1918] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:29:30.216776 9122 log.cc:1196] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/1ca60f78c34849fa8ccd4e82ef746f61
I20260809 19:29:30.217046 9122 ts_tablet_manager.cc:1939] T 1ca60f78c34849fa8ccd4e82ef746f61 P d9d10408cba64af0b4c2c8085cb668e8: Deleting consensus metadata
I20260809 19:29:30.217630 8580 catalog_manager.cc:4841] TS d9d10408cba64af0b4c2c8085cb668e8 (127.21.81.195:37171): tablet 1ca60f78c34849fa8ccd4e82ef746f61 (table test-table [id=79ac2ce753ac4dd0bebf1ca4f173ffbe]) successfully deleted
W20260809 19:29:30.219436 8577 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:44823 (error 108)
I20260809 19:29:30.219676 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 8800
I20260809 19:29:30.221053 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.219853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:44823 (local address 127.0.0.1:40042)
0809 19:29:30.220965 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:30.221011 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1087}
W20260809 19:29:30.221596 8577 catalog_manager.cc:4584] TS 523a9a06368e4af98323740cf9a15f43 (127.21.81.193:44823): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ca60f78c34849fa8ccd4e82ef746f61: Network error: Client connection negotiation failed: client connection to 127.21.81.193:44823: connect: Connection refused (error 111)
I20260809 19:29:30.232599 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 8944
I20260809 19:29:30.239280 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 8559
2026-08-09T19:29:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1997 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T19:29:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:30Z Disabled control of system clock
I20260809 19:29:30.270507 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:44019
--webserver_interface=127.21.81.254
--webserver_port=0
--builtin_ntp_servers=127.21.81.212:36747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:44019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:29:30.352947 9128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:30.353180 9128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:30.353201 9128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:30.354410 9128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:30.354456 9128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:30.354470 9128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:30.354512 9128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:30.354543 9128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:30.355947 9128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:44019
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:44019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:30.356235 9128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:30.356561 9128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:29:30.361542 9128 server_base.cc:1034] running on GCE node
W20260809 19:29:30.361511 9136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:30.361502 9134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:30.361511 9133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:30.362098 9128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:30.362365 9128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:30.363505 9128 hybrid_clock.cc:648] HybridClock initialized: now 1786303770363492 us; error 30 us; skew 500 ppm
I20260809 19:29:30.364796 9128 webserver.cc:460] Webserver started at http://127.21.81.254:42707/ using document root <none> and password file <none>
I20260809 19:29:30.365043 9128 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:30.365093 9128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:30.365250 9128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:30.366395 9128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "e774632de28643c9965396ef5c75cf78"
format_stamp: "Formatted at 2026-08-09 19:29:30 on dist-test-slave-njxd"
server_key: "f98cf5ce648e92ceda631e7c99bd6723"
server_key_iv: "7fcae583a9ae658563810b4950904c64"
server_key_version: "encryptionkey@0"
I20260809 19:29:30.366770 9128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "e774632de28643c9965396ef5c75cf78"
format_stamp: "Formatted at 2026-08-09 19:29:30 on dist-test-slave-njxd"
server_key: "f98cf5ce648e92ceda631e7c99bd6723"
server_key_iv: "7fcae583a9ae658563810b4950904c64"
server_key_version: "encryptionkey@0"
I20260809 19:29:30.368045 9128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:29:30.368750 9144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:30.368928 9128 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:30.369019 9128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "e774632de28643c9965396ef5c75cf78"
format_stamp: "Formatted at 2026-08-09 19:29:30 on dist-test-slave-njxd"
server_key: "f98cf5ce648e92ceda631e7c99bd6723"
server_key_iv: "7fcae583a9ae658563810b4950904c64"
server_key_version: "encryptionkey@0"
I20260809 19:29:30.369098 9128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:30.394802 9128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:30.395165 9128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:30.395330 9128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:30.399586 9128 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:44019
I20260809 19:29:30.399631 9202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:44019 every 8 connection(s)
I20260809 19:29:30.400003 9128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:30.401001 9203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:30.403340 9203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78: Bootstrap starting.
I20260809 19:29:30.403870 9203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:30.404111 9203 log.cc:824] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:30.404745 9203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78: No bootstrap required, opened a new log
I20260809 19:29:30.405892 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 9128
I20260809 19:29:30.406006 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:30.406193 21831 external_mini_cluster.cc:1428] Setting key d3a6dfe44ea4b8e4f0493456b3974d09
I20260809 19:29:30.406759 9203 raft_consensus.cc:348] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e774632de28643c9965396ef5c75cf78" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 44019 } }
I20260809 19:29:30.406953 9203 raft_consensus.cc:374] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:30.406991 9203 raft_consensus.cc:729] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e774632de28643c9965396ef5c75cf78, State: Initialized, Role: FOLLOWER
I20260809 19:29:30.407114 9203 consensus_queue.cc:260] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e774632de28643c9965396ef5c75cf78" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 44019 } }
I20260809 19:29:30.407188 9203 raft_consensus.cc:388] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:30.407219 9203 raft_consensus.cc:482] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:30.407274 9203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:30.407959 9203 raft_consensus.cc:504] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e774632de28643c9965396ef5c75cf78" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 44019 } }
I20260809 19:29:30.408046 9203 leader_election.cc:302] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e774632de28643c9965396ef5c75cf78; no voters:
I20260809 19:29:30.408216 9203 leader_election.cc:288] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:30.408268 9208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:30.408449 9208 raft_consensus.cc:686] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [term 1 LEADER]: Becoming Leader. State: Replica: e774632de28643c9965396ef5c75cf78, State: Running, Role: LEADER
I20260809 19:29:30.408488 9203 sys_catalog.cc:564] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:30.408574 9208 consensus_queue.cc:237] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e774632de28643c9965396ef5c75cf78" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 44019 } }
I20260809 19:29:30.408915 9208 sys_catalog.cc:455] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e774632de28643c9965396ef5c75cf78. Latest consensus state: current_term: 1 leader_uuid: "e774632de28643c9965396ef5c75cf78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e774632de28643c9965396ef5c75cf78" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 44019 } } }
I20260809 19:29:30.409015 9208 sys_catalog.cc:458] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:30.408932 9209 sys_catalog.cc:455] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e774632de28643c9965396ef5c75cf78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e774632de28643c9965396ef5c75cf78" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 44019 } } }
I20260809 19:29:30.409112 9209 sys_catalog.cc:458] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:30.409667 9216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:30.410256 9216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:30.413214 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.406758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54772 (local address 127.21.81.254:44019)
0809 19:29:30.407011 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.407020 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.407053 (+ 33us) server_negotiation.cc:407] Connection header received
0809 19:29:30.407164 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.407172 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.407244 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.407391 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:30.407649 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.409616 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.411315 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.411698 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.411830 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.412562 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:30.412566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:30.412566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:30.412743 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:30.412775 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:30.412785 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:30.412864 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.412941 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.413022 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":113,"threads_started":1}
I20260809 19:29:30.413976 9216 catalog_manager.cc:1347] Generated new cluster ID: 69e28069584b4e35895fdf742d82b3d8
I20260809 19:29:30.414026 9216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:30.426151 9216 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:30.426898 9216 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:30.435268 9216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78: Generated new TSK 0
I20260809 19:29:30.435488 9216 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:30.437359 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:44019
--builtin_ntp_servers=127.21.81.212:36747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:29:30.521438 9227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:30.521730 9227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:30.521775 9227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:30.523061 9227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:30.523125 9227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:30.523172 9227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:29:30.524637 9227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:44019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:30.524925 9227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:30.525259 9227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:30.531580 9232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:30.531595 9233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:30.531829 9235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:30.531821 9227 server_base.cc:1034] running on GCE node
I20260809 19:29:30.532045 9227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:30.532331 9227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:30.533488 9227 hybrid_clock.cc:648] HybridClock initialized: now 1786303770533453 us; error 52 us; skew 500 ppm
I20260809 19:29:30.534866 9227 webserver.cc:460] Webserver started at http://127.21.81.193:42851/ using document root <none> and password file <none>
I20260809 19:29:30.535192 9227 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:30.535262 9227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:30.535401 9227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:30.536485 9227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "18c5e2a07e86479a9884b2f94015f241"
format_stamp: "Formatted at 2026-08-09 19:29:30 on dist-test-slave-njxd"
server_key: "b3c72b97e28077530b6af8527debe03d"
server_key_iv: "ac070388458c80d60775360980302c8a"
server_key_version: "encryptionkey@0"
I20260809 19:29:30.536871 9227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "18c5e2a07e86479a9884b2f94015f241"
format_stamp: "Formatted at 2026-08-09 19:29:30 on dist-test-slave-njxd"
server_key: "b3c72b97e28077530b6af8527debe03d"
server_key_iv: "ac070388458c80d60775360980302c8a"
server_key_version: "encryptionkey@0"
I20260809 19:29:30.538357 9227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:29:30.539192 9241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:30.539379 9227 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:29:30.539469 9227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "18c5e2a07e86479a9884b2f94015f241"
format_stamp: "Formatted at 2026-08-09 19:29:30 on dist-test-slave-njxd"
server_key: "b3c72b97e28077530b6af8527debe03d"
server_key_iv: "ac070388458c80d60775360980302c8a"
server_key_version: "encryptionkey@0"
I20260809 19:29:30.539558 9227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:30.550464 9227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:30.550792 9227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:30.550946 9227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:30.551180 9227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:30.551512 9227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:30.551566 9227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:30.551610 9227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:30.551640 9227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:30.557670 9227 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:46143
I20260809 19:29:30.557771 9364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:46143 every 8 connection(s)
I20260809 19:29:30.558116 9227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:29:30.563048 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 9227
I20260809 19:29:30.563172 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:29:30.563376 21831 external_mini_cluster.cc:1428] Setting key 99ed01bdc8aa5d792140d27857c1ca17
I20260809 19:29:30.564220 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.558883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:50889 (local address 127.21.81.254:44019)
0809 19:29:30.558961 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.558963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.559182 (+ 219us) server_negotiation.cc:407] Connection header received
0809 19:29:30.559693 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.559694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.559707 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.559762 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:30.560558 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.561489 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.562566 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.562833 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.562918 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.563906 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:30.563907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:30.563907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:30.563914 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:30.563929 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:30.563934 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:30.564007 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.564091 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.564122 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 19:29:30.564477 9367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.558878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:44019 (local address 127.21.81.193:50889)
0809 19:29:30.559076 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:30.559111 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:30.559599 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:30.559837 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:30.559840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:30.559855 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:30.560368 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.560372 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.561625 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.561626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.562448 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.562451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.562933 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.562934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.563079 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.563755 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:29:30.563755 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:30.563793 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:29:30.563822 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:30.564007 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:30.564012 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:30.564015 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:29:30.564089 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 19:29:30.564223 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":90,"threads_started":1}
I20260809 19:29:30.564509 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:44019
--builtin_ntp_servers=127.21.81.212:36747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 19:29:30.565074 9365 heartbeater.cc:347] Connected to a master server at 127.21.81.254:44019
I20260809 19:29:30.565209 9365 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:30.565403 9365 heartbeater.cc:511] Master 127.21.81.254:44019 requested a full tablet report, sending...
I20260809 19:29:30.566005 9162 ts_manager.cc:194] Registered new tserver with Master: 18c5e2a07e86479a9884b2f94015f241 (127.21.81.193:46143)
I20260809 19:29:30.566656 9162 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.81.193:50889
W20260809 19:29:30.648859 9368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:30.649093 9368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:30.649138 9368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:30.650370 9368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:30.650435 9368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:30.650475 9368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:29:30.651849 9368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:44019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:30.652153 9368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:30.652465 9368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:30.657541 9373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:30.657706 9368 server_base.cc:1034] running on GCE node
W20260809 19:29:30.657744 9376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:30.657516 9374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:30.658002 9368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:30.658250 9368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:30.659417 9368 hybrid_clock.cc:648] HybridClock initialized: now 1786303770659404 us; error 28 us; skew 500 ppm
I20260809 19:29:30.660840 9368 webserver.cc:460] Webserver started at http://127.21.81.194:34979/ using document root <none> and password file <none>
I20260809 19:29:30.661092 9368 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:30.661144 9368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:30.661309 9368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:30.662461 9368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "578fed5478174e3dbbdea4dde0108c02"
format_stamp: "Formatted at 2026-08-09 19:29:30 on dist-test-slave-njxd"
server_key: "78de21fab33b676a8609e8b9a9419c07"
server_key_iv: "2c2938f87115dab7d8973bda38b31a51"
server_key_version: "encryptionkey@0"
I20260809 19:29:30.662832 9368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "578fed5478174e3dbbdea4dde0108c02"
format_stamp: "Formatted at 2026-08-09 19:29:30 on dist-test-slave-njxd"
server_key: "78de21fab33b676a8609e8b9a9419c07"
server_key_iv: "2c2938f87115dab7d8973bda38b31a51"
server_key_version: "encryptionkey@0"
I20260809 19:29:30.664152 9368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:29:30.664907 9382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:30.665091 9368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:29:30.665198 9368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "578fed5478174e3dbbdea4dde0108c02"
format_stamp: "Formatted at 2026-08-09 19:29:30 on dist-test-slave-njxd"
server_key: "78de21fab33b676a8609e8b9a9419c07"
server_key_iv: "2c2938f87115dab7d8973bda38b31a51"
server_key_version: "encryptionkey@0"
I20260809 19:29:30.665288 9368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:30.685664 9368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:30.686039 9368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:30.686214 9368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:30.686465 9368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:30.686842 9368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:30.686908 9368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:30.686950 9368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:30.686980 9368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:30.693082 9368 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:33425
I20260809 19:29:30.693141 9507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:33425 every 8 connection(s)
I20260809 19:29:30.693534 9368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:29:30.698557 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.694305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:46517 (local address 127.21.81.254:44019)
0809 19:29:30.694374 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.694376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.694633 (+ 257us) server_negotiation.cc:407] Connection header received
0809 19:29:30.695109 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.695110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.695119 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.695185 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:30.695858 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.696356 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.697281 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.697453 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.697501 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.698332 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:30.698334 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:30.698334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:30.698338 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:30.698350 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:30.698354 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:30.698396 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.698475 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.698499 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:29:30.698751 9510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.694289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:44019 (local address 127.21.81.194:46517)
0809 19:29:30.694481 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:30.694525 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:30.695026 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:30.695257 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:30.695260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:30.695269 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:30.695781 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.695785 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.696440 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.696440 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.697209 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.697211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.697525 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.697526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.697711 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.698206 (+ 495us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:29:30.698207 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:30.698233 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:29:30.698254 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:30.698408 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:30.698413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:30.698415 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:29:30.698465 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 19:29:30.698542 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260809 19:29:30.699412 9508 heartbeater.cc:347] Connected to a master server at 127.21.81.254:44019
I20260809 19:29:30.699505 9508 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:30.699658 9508 heartbeater.cc:511] Master 127.21.81.254:44019 requested a full tablet report, sending...
I20260809 19:29:30.700009 9162 ts_manager.cc:194] Registered new tserver with Master: 578fed5478174e3dbbdea4dde0108c02 (127.21.81.194:33425)
I20260809 19:29:30.700428 9162 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.81.194:46517
I20260809 19:29:30.700482 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 9368
I20260809 19:29:30.700592 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:29:30.700852 21831 external_mini_cluster.cc:1428] Setting key 52f40bd099114d40ac23c293836bb62d
I20260809 19:29:30.701810 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:44019
--builtin_ntp_servers=127.21.81.212:36747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:29:30.785177 9511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:30.785432 9511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:30.785471 9511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:30.786680 9511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:30.786758 9511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:30.786803 9511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:29:30.788152 9511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:44019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:30.788427 9511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:30.788724 9511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:30.793776 9519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:30.793766 9516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:30.793833 9511 server_base.cc:1034] running on GCE node
W20260809 19:29:30.793762 9517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:30.794193 9511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:30.794486 9511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:30.795650 9511 hybrid_clock.cc:648] HybridClock initialized: now 1786303770795656 us; error 47 us; skew 500 ppm
I20260809 19:29:30.796898 9511 webserver.cc:460] Webserver started at http://127.21.81.195:43215/ using document root <none> and password file <none>
I20260809 19:29:30.797149 9511 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:30.797220 9511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:30.797381 9511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:30.798501 9511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "8492f00c41ab4507861d9507f54f295b"
format_stamp: "Formatted at 2026-08-09 19:29:30 on dist-test-slave-njxd"
server_key: "3dc45b44b6bb1fb20bcc788d95377304"
server_key_iv: "bed21d6de2ad0bdfcf83b97bd99a0849"
server_key_version: "encryptionkey@0"
I20260809 19:29:30.798880 9511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "8492f00c41ab4507861d9507f54f295b"
format_stamp: "Formatted at 2026-08-09 19:29:30 on dist-test-slave-njxd"
server_key: "3dc45b44b6bb1fb20bcc788d95377304"
server_key_iv: "bed21d6de2ad0bdfcf83b97bd99a0849"
server_key_version: "encryptionkey@0"
I20260809 19:29:30.800200 9511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:29:30.800915 9525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:30.801086 9511 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:30.801199 9511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "8492f00c41ab4507861d9507f54f295b"
format_stamp: "Formatted at 2026-08-09 19:29:30 on dist-test-slave-njxd"
server_key: "3dc45b44b6bb1fb20bcc788d95377304"
server_key_iv: "bed21d6de2ad0bdfcf83b97bd99a0849"
server_key_version: "encryptionkey@0"
I20260809 19:29:30.801273 9511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:30.812937 9511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:30.813213 9511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:30.813359 9511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:30.813616 9511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:30.813951 9511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:30.814004 9511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:30.814043 9511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:30.814069 9511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:30.819768 9511 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:35677
I20260809 19:29:30.819870 9645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:35677 every 8 connection(s)
I20260809 19:29:30.820166 9511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:29:30.825500 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.820923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:43577 (local address 127.21.81.254:44019)
0809 19:29:30.820990 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.820992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.821245 (+ 253us) server_negotiation.cc:407] Connection header received
0809 19:29:30.821806 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.821807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.821819 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.821884 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:30.822564 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.823078 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.824093 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.824297 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.824373 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.825164 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:30.825166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:30.825166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:30.825171 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:30.825187 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:30.825191 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:30.825268 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.825370 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.825405 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:29:30.825800 9649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.820905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:44019 (local address 127.21.81.195:43577)
0809 19:29:30.821095 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:30.821150 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:30.821690 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:30.821958 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:30.821961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:30.821969 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:30.822461 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.822465 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.823178 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.823179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.823984 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.823987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.824382 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.824383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.824517 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.825021 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:29:30.825022 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:30.825050 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:29:30.825071 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:30.825270 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:30.825277 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:30.825281 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:29:30.825371 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 19:29:30.825488 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260809 19:29:30.826272 9646 heartbeater.cc:347] Connected to a master server at 127.21.81.254:44019
I20260809 19:29:30.826387 9646 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:30.826589 9646 heartbeater.cc:511] Master 127.21.81.254:44019 requested a full tablet report, sending...
I20260809 19:29:30.826970 9162 ts_manager.cc:194] Registered new tserver with Master: 8492f00c41ab4507861d9507f54f295b (127.21.81.195:35677)
I20260809 19:29:30.827064 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 9511
I20260809 19:29:30.827191 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:29:30.827380 21831 external_mini_cluster.cc:1428] Setting key 17ee716e9c91359821e652a7bf1d592e
I20260809 19:29:30.827425 9162 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.81.195:43577
I20260809 19:29:30.828397 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:29:30.832681 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.829738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54780 (local address 127.21.81.254:44019)
0809 19:29:30.829778 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.829780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.829922 (+ 142us) server_negotiation.cc:407] Connection header received
0809 19:29:30.829995 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.829996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.830010 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.830059 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:30.830240 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.830694 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.831476 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.831682 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.831757 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.832396 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:30.832397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:30.832397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:30.832402 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:30.832416 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:30.832420 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:30.832524 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.832591 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.832601 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:29:30.834177 9162 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:29:30.835707 9162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:29:30.842235 9656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.837992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:33425 (local address 127.0.0.1:53182)
0809 19:29:30.838149 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:30.838188 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:30.838250 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:30.838475 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:30.838478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:30.838481 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:30.838545 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.838547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.839768 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.839769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.841013 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.841016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.841928 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.841929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.842008 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.842021 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:29:30.842098 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 19:29:30.842159 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":57,"threads_started":1}
I20260809 19:29:30.842242 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.838091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53182 (local address 127.21.81.194:33425)
0809 19:29:30.838301 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.838305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.838317 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:29:30.838355 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.838356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.838370 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.838462 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:30.838642 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.839674 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.841106 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.841797 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.841925 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.842024 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.842113 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.842155 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":86,"threads_started":1}
I20260809 19:29:30.842339 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.837999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:35677 (local address 127.0.0.1:60114)
0809 19:29:30.838171 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:30.838208 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:30.838264 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:30.838720 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:30.838721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:30.838723 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:30.838769 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.838771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.840096 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.840097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.841128 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.841130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.842110 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.842111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.842174 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.842178 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:29:30.842209 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 19:29:30.842264 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
I20260809 19:29:30.842654 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.838344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60114 (local address 127.21.81.195:35677)
0809 19:29:30.838547 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.838550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.838559 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:29:30.838597 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.838599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.838612 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.838863 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:30.838921 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.840006 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.841222 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.842001 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.842413 (+ 412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.842511 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.842536 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.842576 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":101,"threads_started":1}
I20260809 19:29:30.842475 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.838185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:46143 (local address 127.0.0.1:46268)
0809 19:29:30.838794 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:30.838804 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:30.838863 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:30.839028 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:30.839028 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:30.839031 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:30.839067 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.839069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.840269 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.840271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.841253 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:30.841256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.842277 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.842277 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:30.842333 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.842336 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:29:30.842386 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 19:29:30.842422 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":90,"threads_started":1}
I20260809 19:29:30.842833 9660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:30.838644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46268 (local address 127.21.81.193:46143)
0809 19:29:30.838879 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:30.838883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:30.838893 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:29:30.838933 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:30.838935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:30.838951 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:30.839029 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:30.839130 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.840182 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.841344 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:30.842095 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:30.842156 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:30.842243 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:30.842539 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 19:29:30.842579 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":95,"threads_started":1}
I20260809 19:29:30.843346 9433 tablet_service.cc:1467] Processing CreateTablet for tablet 6177256371f44580b5e1a4eccd960823 (DEFAULT_TABLE table=test-table [id=1958d18e7446454c861fd667bd4ce037]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:30.843372 9291 tablet_service.cc:1467] Processing CreateTablet for tablet 6177256371f44580b5e1a4eccd960823 (DEFAULT_TABLE table=test-table [id=1958d18e7446454c861fd667bd4ce037]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:30.843348 9577 tablet_service.cc:1467] Processing CreateTablet for tablet 6177256371f44580b5e1a4eccd960823 (DEFAULT_TABLE table=test-table [id=1958d18e7446454c861fd667bd4ce037]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:30.843726 9577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6177256371f44580b5e1a4eccd960823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:30.843726 9291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6177256371f44580b5e1a4eccd960823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:30.843736 9433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6177256371f44580b5e1a4eccd960823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:30.846618 9661 tablet_bootstrap.cc:492] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02: Bootstrap starting.
I20260809 19:29:30.847123 9662 tablet_bootstrap.cc:492] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241: Bootstrap starting.
I20260809 19:29:30.847498 9661 tablet_bootstrap.cc:654] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:30.847818 9662 tablet_bootstrap.cc:654] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:30.847836 9661 log.cc:824] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:30.848165 9662 log.cc:824] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:30.848515 9663 tablet_bootstrap.cc:492] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b: Bootstrap starting.
I20260809 19:29:30.848727 9661 tablet_bootstrap.cc:492] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02: No bootstrap required, opened a new log
I20260809 19:29:30.848860 9661 ts_tablet_manager.cc:1397] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:29:30.848935 9662 tablet_bootstrap.cc:492] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241: No bootstrap required, opened a new log
I20260809 19:29:30.849009 9662 ts_tablet_manager.cc:1397] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:29:30.849197 9663 tablet_bootstrap.cc:654] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:30.849541 9663 log.cc:824] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:30.850461 9663 tablet_bootstrap.cc:492] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b: No bootstrap required, opened a new log
I20260809 19:29:30.850535 9663 ts_tablet_manager.cc:1397] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:29:30.851266 9661 raft_consensus.cc:348] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578fed5478174e3dbbdea4dde0108c02" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33425 } } peers { permanent_uuid: "8492f00c41ab4507861d9507f54f295b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 35677 } } peers { permanent_uuid: "18c5e2a07e86479a9884b2f94015f241" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46143 } }
I20260809 19:29:30.851421 9661 raft_consensus.cc:374] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:30.851467 9661 raft_consensus.cc:729] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 578fed5478174e3dbbdea4dde0108c02, State: Initialized, Role: FOLLOWER
I20260809 19:29:30.851579 9661 consensus_queue.cc:260] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578fed5478174e3dbbdea4dde0108c02" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33425 } } peers { permanent_uuid: "8492f00c41ab4507861d9507f54f295b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 35677 } } peers { permanent_uuid: "18c5e2a07e86479a9884b2f94015f241" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46143 } }
I20260809 19:29:30.851884 9661 ts_tablet_manager.cc:1428] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:29:30.851919 9508 heartbeater.cc:503] Master 127.21.81.254:44019 was elected leader, sending a full tablet report...
I20260809 19:29:30.851763 9662 raft_consensus.cc:348] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578fed5478174e3dbbdea4dde0108c02" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33425 } } peers { permanent_uuid: "8492f00c41ab4507861d9507f54f295b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 35677 } } peers { permanent_uuid: "18c5e2a07e86479a9884b2f94015f241" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46143 } }
I20260809 19:29:30.852319 9662 raft_consensus.cc:374] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:30.852373 9662 raft_consensus.cc:729] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18c5e2a07e86479a9884b2f94015f241, State: Initialized, Role: FOLLOWER
I20260809 19:29:30.852495 9662 consensus_queue.cc:260] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578fed5478174e3dbbdea4dde0108c02" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33425 } } peers { permanent_uuid: "8492f00c41ab4507861d9507f54f295b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 35677 } } peers { permanent_uuid: "18c5e2a07e86479a9884b2f94015f241" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46143 } }
I20260809 19:29:30.852756 9662 ts_tablet_manager.cc:1428] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 19:29:30.852901 9365 heartbeater.cc:503] Master 127.21.81.254:44019 was elected leader, sending a full tablet report...
I20260809 19:29:30.853266 9663 raft_consensus.cc:348] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578fed5478174e3dbbdea4dde0108c02" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33425 } } peers { permanent_uuid: "8492f00c41ab4507861d9507f54f295b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 35677 } } peers { permanent_uuid: "18c5e2a07e86479a9884b2f94015f241" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46143 } }
I20260809 19:29:30.853451 9663 raft_consensus.cc:374] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:30.853523 9663 raft_consensus.cc:729] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8492f00c41ab4507861d9507f54f295b, State: Initialized, Role: FOLLOWER
I20260809 19:29:30.853678 9663 consensus_queue.cc:260] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578fed5478174e3dbbdea4dde0108c02" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33425 } } peers { permanent_uuid: "8492f00c41ab4507861d9507f54f295b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 35677 } } peers { permanent_uuid: "18c5e2a07e86479a9884b2f94015f241" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46143 } }
I20260809 19:29:30.853952 9663 ts_tablet_manager.cc:1428] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 19:29:30.854022 9646 heartbeater.cc:503] Master 127.21.81.254:44019 was elected leader, sending a full tablet report...
W20260809 19:29:30.944113 9509 tablet.cc:2369] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:29:31.018981 9668 raft_consensus.cc:482] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:29:31.019126 9668 raft_consensus.cc:504] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578fed5478174e3dbbdea4dde0108c02" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33425 } } peers { permanent_uuid: "8492f00c41ab4507861d9507f54f295b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 35677 } } peers { permanent_uuid: "18c5e2a07e86479a9884b2f94015f241" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46143 } }
I20260809 19:29:31.019479 9668 leader_election.cc:288] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 578fed5478174e3dbbdea4dde0108c02 (127.21.81.194:33425), 8492f00c41ab4507861d9507f54f295b (127.21.81.195:35677)
I20260809 19:29:31.023043 9367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:31.019692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:35677 (local address 127.21.81.193:38789)
0809 19:29:31.019756 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:31.019773 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:31.019845 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:31.020059 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:31.020060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:31.020062 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:31.020160 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:31.020162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:31.020874 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:31.020875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:31.022027 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:31.022030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:31.022776 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:31.022777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:31.022847 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:31.022852 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:29:31.022915 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 19:29:31.022962 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":1}
I20260809 19:29:31.023135 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:31.019863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:39213 (local address 127.21.81.194:33425)
0809 19:29:31.019913 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:31.019915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:31.020042 (+ 127us) server_negotiation.cc:407] Connection header received
0809 19:29:31.020090 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:31.020091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:31.020101 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:31.020156 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:31.020341 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:31.020868 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:31.022218 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:31.022791 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:31.022855 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:31.022862 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:31.023016 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 19:29:31.023063 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:29:31.023310 9670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:31.019705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:33425 (local address 127.21.81.193:39213)
0809 19:29:31.019946 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:31.019957 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:31.020028 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:31.020169 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:31.020170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:31.020171 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:31.020270 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:31.020272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:31.020966 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:31.020966 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:31.022127 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:31.022130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:31.022859 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:31.022860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:31.022945 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:31.022949 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:29:31.023007 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 19:29:31.023046 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":13,"thread_start_us":139,"threads_started":1}
I20260809 19:29:31.023757 9456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6177256371f44580b5e1a4eccd960823" candidate_uuid: "18c5e2a07e86479a9884b2f94015f241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "578fed5478174e3dbbdea4dde0108c02" is_pre_election: true
I20260809 19:29:31.023929 9456 raft_consensus.cc:2454] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18c5e2a07e86479a9884b2f94015f241 in term 0.
I20260809 19:29:31.024050 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:31.019865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:38789 (local address 127.21.81.195:35677)
0809 19:29:31.019915 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:31.019918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:31.019930 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:29:31.019967 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:31.019968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:31.019979 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:31.020039 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:31.020250 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:31.020771 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:31.022116 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:31.022683 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:31.022760 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:31.022770 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:31.023930 (+ 1160us) server_negotiation.cc:299] Negotiation successful
0809 19:29:31.023981 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:29:31.024421 9598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6177256371f44580b5e1a4eccd960823" candidate_uuid: "18c5e2a07e86479a9884b2f94015f241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8492f00c41ab4507861d9507f54f295b" is_pre_election: true
I20260809 19:29:31.024539 9245 leader_election.cc:302] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18c5e2a07e86479a9884b2f94015f241, 578fed5478174e3dbbdea4dde0108c02; no voters:
I20260809 19:29:31.024585 9598 raft_consensus.cc:2454] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18c5e2a07e86479a9884b2f94015f241 in term 0.
I20260809 19:29:31.024691 9668 raft_consensus.cc:2781] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:29:31.024756 9668 raft_consensus.cc:482] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:29:31.024786 9668 raft_consensus.cc:3037] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:31.025559 9668 raft_consensus.cc:504] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578fed5478174e3dbbdea4dde0108c02" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33425 } } peers { permanent_uuid: "8492f00c41ab4507861d9507f54f295b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 35677 } } peers { permanent_uuid: "18c5e2a07e86479a9884b2f94015f241" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46143 } }
I20260809 19:29:31.025777 9668 leader_election.cc:288] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [CANDIDATE]: Term 1 election: Requested vote from peers 578fed5478174e3dbbdea4dde0108c02 (127.21.81.194:33425), 8492f00c41ab4507861d9507f54f295b (127.21.81.195:35677)
I20260809 19:29:31.025940 9598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6177256371f44580b5e1a4eccd960823" candidate_uuid: "18c5e2a07e86479a9884b2f94015f241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8492f00c41ab4507861d9507f54f295b"
I20260809 19:29:31.025940 9456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6177256371f44580b5e1a4eccd960823" candidate_uuid: "18c5e2a07e86479a9884b2f94015f241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "578fed5478174e3dbbdea4dde0108c02"
I20260809 19:29:31.026019 9598 raft_consensus.cc:3037] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:31.026019 9456 raft_consensus.cc:3037] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:31.026998 9598 raft_consensus.cc:2454] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18c5e2a07e86479a9884b2f94015f241 in term 1.
I20260809 19:29:31.026998 9456 raft_consensus.cc:2454] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18c5e2a07e86479a9884b2f94015f241 in term 1.
I20260809 19:29:31.027237 9245 leader_election.cc:302] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18c5e2a07e86479a9884b2f94015f241, 578fed5478174e3dbbdea4dde0108c02; no voters:
I20260809 19:29:31.027338 9668 raft_consensus.cc:2781] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:31.027494 9668 raft_consensus.cc:686] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [term 1 LEADER]: Becoming Leader. State: Replica: 18c5e2a07e86479a9884b2f94015f241, State: Running, Role: LEADER
I20260809 19:29:31.027611 9668 consensus_queue.cc:237] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578fed5478174e3dbbdea4dde0108c02" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33425 } } peers { permanent_uuid: "8492f00c41ab4507861d9507f54f295b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 35677 } } peers { permanent_uuid: "18c5e2a07e86479a9884b2f94015f241" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46143 } }
I20260809 19:29:31.028561 9162 catalog_manager.cc:5495] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18c5e2a07e86479a9884b2f94015f241 (127.21.81.193). New cstate: current_term: 1 leader_uuid: "18c5e2a07e86479a9884b2f94015f241" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "578fed5478174e3dbbdea4dde0108c02" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8492f00c41ab4507861d9507f54f295b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 35677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18c5e2a07e86479a9884b2f94015f241" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 46143 } health_report { overall_health: HEALTHY } } }
W20260809 19:29:31.058835 9366 tablet.cc:2369] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:29:31.070767 9647 tablet.cc:2369] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:29:31.070820 9660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:31.068285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46284 (local address 127.21.81.193:46143)
0809 19:29:31.068340 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:31.068342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:31.068350 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:29:31.068386 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:31.068387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:31.068408 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:31.068477 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:29:31.068612 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:31.069126 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:31.070035 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:31.070234 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:31.070314 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:31.070455 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:29:31.070548 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:29:31.070668 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:31.070698 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 19:29:31.070709 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:29:31.073002 9598 raft_consensus.cc:1261] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b [term 1 FOLLOWER]: Refusing update from remote peer 18c5e2a07e86479a9884b2f94015f241: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:29:31.073108 9456 raft_consensus.cc:1261] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02 [term 1 FOLLOWER]: Refusing update from remote peer 18c5e2a07e86479a9884b2f94015f241: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:29:31.073382 9668 consensus_queue.cc:1035] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8492f00c41ab4507861d9507f54f295b" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 35677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:29:31.073510 9671 consensus_queue.cc:1035] T 6177256371f44580b5e1a4eccd960823 P 18c5e2a07e86479a9884b2f94015f241 [LEADER]: Connected to new peer: Peer: permanent_uuid: "578fed5478174e3dbbdea4dde0108c02" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 33425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:29:31.076604 9674 mvcc.cc:205] Tried to move back new op lower bound from 7316700246312124416 to 7316700246129676288. Current Snapshot: MvccSnapshot[applied={T|T < 7316700246312124416 or (T in {7316700246312124416})}]
I20260809 19:29:31.077823 9675 mvcc.cc:205] Tried to move back new op lower bound from 7316700246312124416 to 7316700246129676288. Current Snapshot: MvccSnapshot[applied={T|T < 7316700246312124416}]
I20260809 19:29:31.079692 9162 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:29:31.079821 9162 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:29:31.081501 9162 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e774632de28643c9965396ef5c75cf78: Sending DeleteTablet for 3 replicas of tablet 6177256371f44580b5e1a4eccd960823
I20260809 19:29:31.081918 9577 tablet_service.cc:1514] Processing DeleteTablet for tablet 6177256371f44580b5e1a4eccd960823 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:31 UTC) from {username='slave'} at 127.0.0.1:60114
I20260809 19:29:31.081924 9433 tablet_service.cc:1514] Processing DeleteTablet for tablet 6177256371f44580b5e1a4eccd960823 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:31 UTC) from {username='slave'} at 127.0.0.1:53182
I20260809 19:29:31.082180 9291 tablet_service.cc:1514] Processing DeleteTablet for tablet 6177256371f44580b5e1a4eccd960823 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:31 UTC) from {username='slave'} at 127.0.0.1:46268
I20260809 19:29:31.082310 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 9227
I20260809 19:29:31.082329 9685 tablet_replica.cc:330] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b: stopping tablet replica
I20260809 19:29:31.083477 9684 tablet_replica.cc:330] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02: stopping tablet replica
I20260809 19:29:31.083588 9684 raft_consensus.cc:2229] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:29:31.083698 9684 raft_consensus.cc:2258] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:29:31.084376 9684 ts_tablet_manager.cc:1905] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:29:31.084511 9685 raft_consensus.cc:2229] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:29:31.084649 9685 raft_consensus.cc:2258] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:29:31.085037 9685 ts_tablet_manager.cc:1905] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:29:31.086839 9685 ts_tablet_manager.cc:1918] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:29:31.086870 9684 ts_tablet_manager.cc:1918] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:29:31.086910 9685 log.cc:1196] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/6177256371f44580b5e1a4eccd960823
I20260809 19:29:31.086956 9684 log.cc:1196] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/6177256371f44580b5e1a4eccd960823
I20260809 19:29:31.087133 9685 ts_tablet_manager.cc:1939] T 6177256371f44580b5e1a4eccd960823 P 8492f00c41ab4507861d9507f54f295b: Deleting consensus metadata
I20260809 19:29:31.087271 9684 ts_tablet_manager.cc:1939] T 6177256371f44580b5e1a4eccd960823 P 578fed5478174e3dbbdea4dde0108c02: Deleting consensus metadata
I20260809 19:29:31.087733 9149 catalog_manager.cc:4841] TS 578fed5478174e3dbbdea4dde0108c02 (127.21.81.194:33425): tablet 6177256371f44580b5e1a4eccd960823 (table test-table [id=1958d18e7446454c861fd667bd4ce037]) successfully deleted
I20260809 19:29:31.089919 9145 catalog_manager.cc:4841] TS 8492f00c41ab4507861d9507f54f295b (127.21.81.195:35677): tablet 6177256371f44580b5e1a4eccd960823 (table test-table [id=1958d18e7446454c861fd667bd4ce037]) successfully deleted
W20260809 19:29:31.094547 9149 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:46143 (error 108)
I20260809 19:29:31.095149 9656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:31.094997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:46143 (local address 127.0.0.1:46296)
0809 19:29:31.095056 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:31.095094 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:46143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 19:29:31.095196 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 9368
W20260809 19:29:31.095299 9149 catalog_manager.cc:4584] TS 18c5e2a07e86479a9884b2f94015f241 (127.21.81.193:46143): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6177256371f44580b5e1a4eccd960823: Network error: Client connection negotiation failed: client connection to 127.21.81.193:46143: connect: Connection refused (error 111)
I20260809 19:29:31.101958 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 9511
I20260809 19:29:31.107722 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 9128
2026-08-09T19:29:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (867 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:31 dist-test-slave-njxd krb5kdc[9691](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:31 dist-test-slave-njxd krb5kdc[9691](info): set up 2 sockets
Aug 09 19:29:31 dist-test-slave-njxd krb5kdc[9691](info): commencing operation
krb5kdc: starting...
W20260809 19:29:32.161175 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:32 dist-test-slave-njxd krb5kdc[9691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:29:32.282497 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:38427
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:38427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:32.364526 9704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:32.364723 9704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:32.364745 9704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:32.365952 9704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:32.365993 9704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:32.366006 9704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:32.366019 9704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:32.366071 9704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:32.367513 9704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:38427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:38427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:32.367787 9704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:32.368021 9704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:32.372869 9709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:32.372977 9704 server_base.cc:1034] running on GCE node
W20260809 19:29:32.372862 9712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:32.372862 9710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:32.373371 9704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:32.373713 9704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:32.374882 9704 hybrid_clock.cc:648] HybridClock initialized: now 1786303772374871 us; error 39 us; skew 500 ppm
Aug 09 19:29:32 dist-test-slave-njxd krb5kdc[9691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303772, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:32.377303 9704 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:29:32.377805 9704 webserver.cc:460] Webserver started at http://127.21.81.254:40821/ using document root <none> and password file <none>
I20260809 19:29:32.378024 9704 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:32.378099 9704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:32.378257 9704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:32.379290 9704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "7d1dd5d2b7734c64a02c0f3a1a227d76"
format_stamp: "Formatted at 2026-08-09 19:29:32 on dist-test-slave-njxd"
server_key: "88adf15999f88209ce2248766e87e8b4"
server_key_iv: "cb73e86eeb5cef82accdbc097785aa1f"
server_key_version: "encryptionkey@0"
I20260809 19:29:32.379655 9704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "7d1dd5d2b7734c64a02c0f3a1a227d76"
format_stamp: "Formatted at 2026-08-09 19:29:32 on dist-test-slave-njxd"
server_key: "88adf15999f88209ce2248766e87e8b4"
server_key_iv: "cb73e86eeb5cef82accdbc097785aa1f"
server_key_version: "encryptionkey@0"
I20260809 19:29:32.381106 9704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:29:32.381896 9719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:32.382071 9704 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:29:32.382174 9704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "7d1dd5d2b7734c64a02c0f3a1a227d76"
format_stamp: "Formatted at 2026-08-09 19:29:32 on dist-test-slave-njxd"
server_key: "88adf15999f88209ce2248766e87e8b4"
server_key_iv: "cb73e86eeb5cef82accdbc097785aa1f"
server_key_version: "encryptionkey@0"
I20260809 19:29:32.382261 9704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:32.395257 9704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:32.396926 9704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:32.397071 9704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:32.401187 9704 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:38427
I20260809 19:29:32.401221 9775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:38427 every 8 connection(s)
I20260809 19:29:32.401635 9704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:32.402469 9776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:32.404685 9776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76: Bootstrap starting.
I20260809 19:29:32.405296 9776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:32.405647 9776 log.cc:824] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:32.406324 9776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76: No bootstrap required, opened a new log
I20260809 19:29:32.407976 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 9704
I20260809 19:29:32.407924 9776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1dd5d2b7734c64a02c0f3a1a227d76" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 38427 } }
I20260809 19:29:32.408044 9776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:32.408089 9776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d1dd5d2b7734c64a02c0f3a1a227d76, State: Initialized, Role: FOLLOWER
I20260809 19:29:32.408108 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:32.408237 9776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1dd5d2b7734c64a02c0f3a1a227d76" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 38427 } }
I20260809 19:29:32.408301 21831 external_mini_cluster.cc:1428] Setting key a287db73b3d2a823e408625c44adc29e
I20260809 19:29:32.408325 9776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:32.408350 9776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:32.408416 9776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:32.409545 9776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1dd5d2b7734c64a02c0f3a1a227d76" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 38427 } }
I20260809 19:29:32.409718 9776 leader_election.cc:302] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d1dd5d2b7734c64a02c0f3a1a227d76; no voters:
I20260809 19:29:32.409885 9776 leader_election.cc:288] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:32.409937 9781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:32.410082 9781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [term 1 LEADER]: Becoming Leader. State: Replica: 7d1dd5d2b7734c64a02c0f3a1a227d76, State: Running, Role: LEADER
I20260809 19:29:32.410161 9776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:32.410176 9781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1dd5d2b7734c64a02c0f3a1a227d76" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 38427 } }
I20260809 19:29:32.410561 9781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d1dd5d2b7734c64a02c0f3a1a227d76. Latest consensus state: current_term: 1 leader_uuid: "7d1dd5d2b7734c64a02c0f3a1a227d76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1dd5d2b7734c64a02c0f3a1a227d76" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 38427 } } }
I20260809 19:29:32.410645 9781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:32.410574 9783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d1dd5d2b7734c64a02c0f3a1a227d76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d1dd5d2b7734c64a02c0f3a1a227d76" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 38427 } } }
I20260809 19:29:32.410851 9783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:32.410862 9788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:32.411538 9788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:32.413331 9788 catalog_manager.cc:1347] Generated new cluster ID: 1f36b22f04a14017a0e14e11e91d60bb
I20260809 19:29:32.413386 9788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:29:32 dist-test-slave-njxd krb5kdc[9691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:29:32.419080 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:32.408836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50638 (local address 127.21.81.254:38427)
0809 19:29:32.409062 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:32.409074 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:32.409117 (+ 43us) server_negotiation.cc:407] Connection header received
0809 19:29:32.409264 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:32.409271 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:32.409327 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:32.409476 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:32.410356 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.411776 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.412893 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.413361 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.413483 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:32.415939 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:32.415943 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:32.415943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:32.415954 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:32.417489 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:32.417949 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:32.417952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:32.417952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:32.418063 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:32.418283 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:32.418284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:32.418284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:32.418525 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:32.418694 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:32.418770 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 19:29:32.418908 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":108,"threads_started":1}
I20260809 19:29:32.427055 9788 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:32.427568 9788 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:32.437743 9788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76: Generated new TSK 0
I20260809 19:29:32.437955 9788 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.81.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
Entry for principal kudu/127.21.81.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab.
I20260809 19:29:32.470676 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:38427
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:32.557822 9803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:32.558048 9803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:32.558089 9803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:32.559813 9803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:32.559885 9803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:32.559931 9803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:29:32.561422 9803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.193.keytab
--principal=kudu/127.21.81.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:38427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:32.561719 9803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:32.561959 9803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:29:32.566854 9803 server_base.cc:1034] running on GCE node
W20260809 19:29:32.566877 9811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:32.566887 9808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:32.566980 9809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:32.567212 9803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:32.567502 9803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:32.568655 9803 hybrid_clock.cc:648] HybridClock initialized: now 1786303772568639 us; error 36 us; skew 500 ppm
Aug 09 19:29:32 dist-test-slave-njxd krb5kdc[9691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303772, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:32.571200 9803 init.cc:377] Logged in from keytab as kudu/127.21.81.193@KRBTEST.COM (short username kudu)
I20260809 19:29:32.571645 9803 webserver.cc:460] Webserver started at http://127.21.81.193:43261/ using document root <none> and password file <none>
I20260809 19:29:32.571885 9803 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:32.571967 9803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:32.572120 9803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:32.573127 9803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "dc3fdba3ca934282b221ea627a7f6b50"
format_stamp: "Formatted at 2026-08-09 19:29:32 on dist-test-slave-njxd"
server_key: "83b059f5492ec03557f16a1af39852c3"
server_key_iv: "1b9f13e34b4524fb122d370c56513fc4"
server_key_version: "encryptionkey@0"
I20260809 19:29:32.573482 9803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "dc3fdba3ca934282b221ea627a7f6b50"
format_stamp: "Formatted at 2026-08-09 19:29:32 on dist-test-slave-njxd"
server_key: "83b059f5492ec03557f16a1af39852c3"
server_key_iv: "1b9f13e34b4524fb122d370c56513fc4"
server_key_version: "encryptionkey@0"
I20260809 19:29:32.574896 9803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:29:32.575649 9818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:32.575839 9803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:29:32.575940 9803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "dc3fdba3ca934282b221ea627a7f6b50"
format_stamp: "Formatted at 2026-08-09 19:29:32 on dist-test-slave-njxd"
server_key: "83b059f5492ec03557f16a1af39852c3"
server_key_iv: "1b9f13e34b4524fb122d370c56513fc4"
server_key_version: "encryptionkey@0"
I20260809 19:29:32.576020 9803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:32.596112 9803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:32.598487 9803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:32.598658 9803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:32.598912 9803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:32.599228 9803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:32.599283 9803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:32.599335 9803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:32.599359 9803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:32.605290 9803 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:35839
I20260809 19:29:32.605341 9936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:35839 every 8 connection(s)
I20260809 19:29:32.605742 9803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:29:32.606837 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 9803
I20260809 19:29:32.606962 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:29:32.607160 21831 external_mini_cluster.cc:1428] Setting key a99a73df6304ea1f7ddb4030d9b278e9
Aug 09 19:29:32 dist-test-slave-njxd krb5kdc[9691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303772, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.193@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.81.194@KRBTEST.COM; defaulting to no policy
I20260809 19:29:32.616582 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:32.606517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:38467 (local address 127.21.81.254:38427)
0809 19:29:32.606589 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:32.606591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:32.606885 (+ 294us) server_negotiation.cc:407] Connection header received
0809 19:29:32.607576 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:32.607577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:32.607590 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:32.607645 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:32.608972 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.609480 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.610759 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.611033 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.611091 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:32.613662 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:32.613664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:32.613664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:32.613672 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:32.615151 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:32.615603 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:32.615604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:32.615604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:32.615665 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:32.615931 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:32.615932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:32.615932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:32.616048 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:32.616119 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:32.616407 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 19:29:32.616493 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:29:32.616753 9939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:32.606500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:38427 (local address 127.21.81.193:38467)
0809 19:29:32.606716 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:32.606767 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:32.607485 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:32.607719 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:32.607723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:32.608574 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:32.608877 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:32.608883 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.609604 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.609606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:32.610676 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:32.610679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.611136 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.611138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:32.611323 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:32.611813 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:32.611824 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:32.613561 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:32.615282 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:32.615289 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:32.615292 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:32.615515 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:32.615738 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:32.615738 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:32.615739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:32.615851 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:32.616124 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:32.616131 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:32.616272 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 19:29:32.616372 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 19:29:32.616526 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":90,"threads_started":1}
I20260809 19:29:32.617287 9937 heartbeater.cc:347] Connected to a master server at 127.21.81.254:38427
I20260809 19:29:32.617412 9937 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:32.617636 9937 heartbeater.cc:511] Master 127.21.81.254:38427 requested a full tablet report, sending...
I20260809 19:29:32.618191 9737 ts_manager.cc:194] Registered new tserver with Master: dc3fdba3ca934282b221ea627a7f6b50 (127.21.81.193:35839)
I20260809 19:29:32.618793 9737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.193@KRBTEST.COM'} at 127.21.81.193:38467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
Entry for principal kudu/127.21.81.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab.
I20260809 19:29:32.632400 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:38427
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:32.714531 9942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:32.714757 9942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:32.714782 9942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:32.715953 9942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:32.715996 9942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:32.716046 9942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:29:32.717623 9942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.194.keytab
--principal=kudu/127.21.81.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:38427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:32.717890 9942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:32.718116 9942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:32.723126 9952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:32.723208 9949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:32.723141 9942 server_base.cc:1034] running on GCE node
W20260809 19:29:32.723114 9948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:32.723471 9942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:32.723762 9942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:32.724927 9942 hybrid_clock.cc:648] HybridClock initialized: now 1786303772724915 us; error 43 us; skew 500 ppm
Aug 09 19:29:32 dist-test-slave-njxd krb5kdc[9691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303772, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:32.727496 9942 init.cc:377] Logged in from keytab as kudu/127.21.81.194@KRBTEST.COM (short username kudu)
I20260809 19:29:32.727878 9942 webserver.cc:460] Webserver started at http://127.21.81.194:42733/ using document root <none> and password file <none>
I20260809 19:29:32.728052 9942 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:32.728091 9942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:32.728195 9942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:32.729149 9942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "e08536806fa94af2bb41eb785e5422f6"
format_stamp: "Formatted at 2026-08-09 19:29:32 on dist-test-slave-njxd"
server_key: "cfb71625c05ce59c1ea50df7a92952ea"
server_key_iv: "dfcff1c0ffbdefdb916610da3cb2ddfb"
server_key_version: "encryptionkey@0"
I20260809 19:29:32.729487 9942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "e08536806fa94af2bb41eb785e5422f6"
format_stamp: "Formatted at 2026-08-09 19:29:32 on dist-test-slave-njxd"
server_key: "cfb71625c05ce59c1ea50df7a92952ea"
server_key_iv: "dfcff1c0ffbdefdb916610da3cb2ddfb"
server_key_version: "encryptionkey@0"
I20260809 19:29:32.730829 9942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:29:32.731487 9959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:32.731660 9942 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:29:32.731753 9942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "e08536806fa94af2bb41eb785e5422f6"
format_stamp: "Formatted at 2026-08-09 19:29:32 on dist-test-slave-njxd"
server_key: "cfb71625c05ce59c1ea50df7a92952ea"
server_key_iv: "dfcff1c0ffbdefdb916610da3cb2ddfb"
server_key_version: "encryptionkey@0"
I20260809 19:29:32.731804 9942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:32.744599 9942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:32.746822 9942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:32.746932 9942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:32.747121 9942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:32.747433 9942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:32.747468 9942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:32.747490 9942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:32.747548 9942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:32.753468 9942 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:45371
I20260809 19:29:32.753547 10076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:45371 every 8 connection(s)
I20260809 19:29:32.753890 9942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:29:32.757548 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 9942
I20260809 19:29:32.757704 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:29:32.757896 21831 external_mini_cluster.cc:1428] Setting key e59d3c0fea76cfb6348f27dd830378c0
Aug 09 19:29:32 dist-test-slave-njxd krb5kdc[9691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303772, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.194@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:29:32.764061 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:32.754624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:54427 (local address 127.21.81.254:38427)
0809 19:29:32.754713 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:32.754715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:32.754945 (+ 230us) server_negotiation.cc:407] Connection header received
0809 19:29:32.755477 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:32.755479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:32.755492 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:32.755544 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:32.756499 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.757007 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.758364 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.758566 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.758655 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:32.760998 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:32.761000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:32.761000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:32.761010 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:32.762415 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:32.762932 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:32.762933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:32.762934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:32.762981 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:32.763278 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:32.763279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:32.763280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:32.763420 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:32.763491 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:32.763839 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 19:29:32.763926 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:29:32.764240 10079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:32.754583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:38427 (local address 127.21.81.194:54427)
0809 19:29:32.754793 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:32.754855 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:32.755361 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:32.755613 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:32.755617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:32.756195 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:32.756397 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:32.756401 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.757113 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.757114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:32.758272 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:32.758274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.758672 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.758673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:32.758790 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:32.759320 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:32.759328 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:32.760906 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:32.762499 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:32.762506 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:32.762508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:32.762854 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:32.763044 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:32.763045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:32.763046 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:32.763194 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:32.763497 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:32.763504 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:32.763670 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 19:29:32.763795 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 19:29:32.763973 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":97,"threads_started":1}
I20260809 19:29:32.764904 10077 heartbeater.cc:347] Connected to a master server at 127.21.81.254:38427
I20260809 19:29:32.765024 10077 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:32.765259 10077 heartbeater.cc:511] Master 127.21.81.254:38427 requested a full tablet report, sending...
I20260809 19:29:32.765681 9737 ts_manager.cc:194] Registered new tserver with Master: e08536806fa94af2bb41eb785e5422f6 (127.21.81.194:45371)
I20260809 19:29:32.766081 9737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.194@KRBTEST.COM'} at 127.21.81.194:54427
WARNING: no policy specified for kudu/127.21.81.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
Entry for principal kudu/127.21.81.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab.
I20260809 19:29:32.784948 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:38427
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:32.868161 10082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:32.868404 10082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:32.868427 10082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:32.869621 10082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:32.869681 10082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:32.869711 10082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:29:32.871109 10082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.195.keytab
--principal=kudu/127.21.81.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:38427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:32.871299 10082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:32.871465 10082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:32.876389 10089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:32.876343 10088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:32.876533 10082 server_base.cc:1034] running on GCE node
W20260809 19:29:32.876375 10091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:32.876839 10082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:32.877084 10082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:32.878261 10082 hybrid_clock.cc:648] HybridClock initialized: now 1786303772878240 us; error 34 us; skew 500 ppm
Aug 09 19:29:32 dist-test-slave-njxd krb5kdc[9691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303772, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:32.880900 10082 init.cc:377] Logged in from keytab as kudu/127.21.81.195@KRBTEST.COM (short username kudu)
I20260809 19:29:32.881342 10082 webserver.cc:460] Webserver started at http://127.21.81.195:44175/ using document root <none> and password file <none>
I20260809 19:29:32.881608 10082 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:32.881693 10082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:32.881857 10082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:32.882956 10082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "174eab7bb1544d20b7558eb70253152e"
format_stamp: "Formatted at 2026-08-09 19:29:32 on dist-test-slave-njxd"
server_key: "1961942fac311b0fc76b9caaab343ff8"
server_key_iv: "537e0c371091a0307d218f4232d278ff"
server_key_version: "encryptionkey@0"
I20260809 19:29:32.883265 10082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "174eab7bb1544d20b7558eb70253152e"
format_stamp: "Formatted at 2026-08-09 19:29:32 on dist-test-slave-njxd"
server_key: "1961942fac311b0fc76b9caaab343ff8"
server_key_iv: "537e0c371091a0307d218f4232d278ff"
server_key_version: "encryptionkey@0"
I20260809 19:29:32.884562 10082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:29:32.885324 10098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:32.885476 10082 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:32.885566 10082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "174eab7bb1544d20b7558eb70253152e"
format_stamp: "Formatted at 2026-08-09 19:29:32 on dist-test-slave-njxd"
server_key: "1961942fac311b0fc76b9caaab343ff8"
server_key_iv: "537e0c371091a0307d218f4232d278ff"
server_key_version: "encryptionkey@0"
I20260809 19:29:32.885665 10082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:32.911583 10082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:32.913750 10082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:32.913866 10082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:32.914119 10082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:32.914455 10082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:32.914489 10082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:32.914543 10082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:32.914584 10082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:32.920514 10082 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:37255
I20260809 19:29:32.920610 10219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:37255 every 8 connection(s)
I20260809 19:29:32.920925 10082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:29:32 dist-test-slave-njxd krb5kdc[9691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303772, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.195@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:29:32.929931 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:32.921718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:37209 (local address 127.21.81.254:38427)
0809 19:29:32.921760 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:32.921762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:32.921965 (+ 203us) server_negotiation.cc:407] Connection header received
0809 19:29:32.922456 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:32.922457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:32.922466 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:32.922525 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:32.923468 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.923957 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.924930 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.925107 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.925165 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:32.927407 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:32.927408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:32.927409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:32.927416 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:32.928683 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:32.929049 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:32.929049 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:32.929050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:32.929099 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:32.929352 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:32.929353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:32.929353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:32.929456 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:32.929507 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:32.929799 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 19:29:32.929866 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:29:32.930141 10222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:32.921686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:38427 (local address 127.21.81.195:37209)
0809 19:29:32.921844 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:32.921886 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:32.922374 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:32.922577 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:32.922580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:32.923175 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:32.923381 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:32.923387 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.924042 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.924043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:32.924855 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:32.924857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.925181 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.925182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:32.925286 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:32.925913 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:32.925922 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:32.927313 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:32.928759 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:32.928766 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:32.928768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:32.928985 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:32.929167 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:32.929167 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:32.929168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:32.929298 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:32.929513 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:32.929518 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:32.929682 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 19:29:32.929771 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 19:29:32.929894 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260809 19:29:32.930533 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 10082
I20260809 19:29:32.930646 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:29:32.930717 10220 heartbeater.cc:347] Connected to a master server at 127.21.81.254:38427
I20260809 19:29:32.930827 10220 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:32.930862 21831 external_mini_cluster.cc:1428] Setting key 334bbe05861b3125ed41b680811e15d2
I20260809 19:29:32.931028 10220 heartbeater.cc:511] Master 127.21.81.254:38427 requested a full tablet report, sending...
I20260809 19:29:32.931434 9737 ts_manager.cc:194] Registered new tserver with Master: 174eab7bb1544d20b7558eb70253152e (127.21.81.195:37255)
I20260809 19:29:32.931816 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:29:32.931938 9737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.81.195@KRBTEST.COM'} at 127.21.81.195:37209
I20260809 19:29:32.939745 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:32.933182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50640 (local address 127.21.81.254:38427)
0809 19:29:32.933225 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:32.933227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:32.933357 (+ 130us) server_negotiation.cc:407] Connection header received
0809 19:29:32.933393 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:32.933393 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:32.933402 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:32.933447 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:32.934044 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.934516 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.935624 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.935836 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.935911 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:32.937159 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:32.937160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:32.937161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:32.937168 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:32.938717 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:32.939068 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:32.939069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:32.939069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:32.939121 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:32.939287 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:32.939288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:32.939288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:32.939433 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:32.939508 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:32.939608 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 19:29:32.939671 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:29:32.941340 9737 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:29:32.942950 9737 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:29:32.949447 10234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:32.945303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:37255 (local address 127.0.0.1:39396)
0809 19:29:32.945688 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:32.945761 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:32.945828 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:32.946041 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:32.946045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:32.946048 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:32.946125 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:32.946128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.947190 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.947191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:32.948208 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:32.948211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.949115 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.949116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:32.949220 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:32.949236 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 19:29:32.949312 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 19:29:32.949370 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"spinlock_wait_cycles":384,"thread_start_us":99,"threads_started":1}
I20260809 19:29:32.949467 10235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:32.945497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39396 (local address 127.21.81.195:37255)
0809 19:29:32.945865 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:32.945869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:32.945880 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:29:32.945925 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:32.945927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:32.945939 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:32.946010 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:32.946193 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.947078 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.948307 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.949023 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.949103 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:32.949163 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:32.949343 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 19:29:32.949390 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":119,"threads_started":1}
I20260809 19:29:32.950008 10233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:32.945303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:45371 (local address 127.0.0.1:36920)
0809 19:29:32.945929 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:32.945936 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:32.945964 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:32.946521 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:32.946522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:32.946524 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:32.946568 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:32.946570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.947826 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.947828 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:32.948758 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:32.948760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.949821 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.949822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:32.949891 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:32.949894 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:29:32.949933 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 19:29:32.949963 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":2,"thread_start_us":61,"threads_started":1}
I20260809 19:29:32.950275 10149 tablet_service.cc:1467] Processing CreateTablet for tablet 6f118f9bf51c42ec87dd47db938b3ee0 (DEFAULT_TABLE table=test-table [id=247aee7faba84e61bc5814a42283a390]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:32.950569 10231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:32.945321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:35839 (local address 127.0.0.1:35628)
0809 19:29:32.946036 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:32.946044 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:32.946106 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:32.946393 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:32.946393 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:32.946396 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:32.946450 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:32.946451 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.947767 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.947769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:32.948869 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:32.948872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.950339 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.950340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:32.950408 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:32.950413 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:29:32.950467 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 19:29:32.950504 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":30,"thread_start_us":93,"threads_started":1}
I20260809 19:29:32.950724 10149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f118f9bf51c42ec87dd47db938b3ee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:32.950881 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:32.945614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35628 (local address 127.21.81.193:35839)
0809 19:29:32.945810 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:32.945815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:32.946164 (+ 349us) server_negotiation.cc:407] Connection header received
0809 19:29:32.946214 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:32.946216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:32.946231 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:32.946300 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:32.946509 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.947652 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.948945 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.950065 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.950323 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:32.950386 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:32.950560 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 19:29:32.950774 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":126,"threads_started":1}
I20260809 19:29:32.951457 9869 tablet_service.cc:1467] Processing CreateTablet for tablet 6f118f9bf51c42ec87dd47db938b3ee0 (DEFAULT_TABLE table=test-table [id=247aee7faba84e61bc5814a42283a390]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:32.951761 9869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f118f9bf51c42ec87dd47db938b3ee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:32.953490 10238 tablet_bootstrap.cc:492] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e: Bootstrap starting.
I20260809 19:29:32.953895 10237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:32.945638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36920 (local address 127.21.81.194:45371)
0809 19:29:32.946298 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:32.946302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:32.946312 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:29:32.946348 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:32.946350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:32.946365 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:32.946440 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:32.946659 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.947746 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.948859 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:32.949740 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:32.953632 (+ 3892us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:32.953706 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:32.953746 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 19:29:32.953792 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":112,"threads_started":1}
I20260809 19:29:32.954290 10238 tablet_bootstrap.cc:654] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:32.954526 10239 tablet_bootstrap.cc:492] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50: Bootstrap starting.
I20260809 19:29:32.954552 10008 tablet_service.cc:1467] Processing CreateTablet for tablet 6f118f9bf51c42ec87dd47db938b3ee0 (DEFAULT_TABLE table=test-table [id=247aee7faba84e61bc5814a42283a390]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:32.954931 10008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f118f9bf51c42ec87dd47db938b3ee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:32.954969 10238 log.cc:824] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:32.955261 10239 tablet_bootstrap.cc:654] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:32.955826 10238 tablet_bootstrap.cc:492] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e: No bootstrap required, opened a new log
I20260809 19:29:32.955876 10239 log.cc:824] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:32.955919 10238 ts_tablet_manager.cc:1397] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:29:32.956696 10239 tablet_bootstrap.cc:492] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50: No bootstrap required, opened a new log
I20260809 19:29:32.956779 10239 ts_tablet_manager.cc:1397] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:29:32.958104 10238 raft_consensus.cc:348] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08536806fa94af2bb41eb785e5422f6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45371 } } peers { permanent_uuid: "174eab7bb1544d20b7558eb70253152e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37255 } } peers { permanent_uuid: "dc3fdba3ca934282b221ea627a7f6b50" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 35839 } }
I20260809 19:29:32.958314 10238 raft_consensus.cc:374] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:32.958359 10238 raft_consensus.cc:729] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 174eab7bb1544d20b7558eb70253152e, State: Initialized, Role: FOLLOWER
I20260809 19:29:32.958482 10238 consensus_queue.cc:260] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08536806fa94af2bb41eb785e5422f6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45371 } } peers { permanent_uuid: "174eab7bb1544d20b7558eb70253152e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37255 } } peers { permanent_uuid: "dc3fdba3ca934282b221ea627a7f6b50" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 35839 } }
I20260809 19:29:32.958767 10238 ts_tablet_manager.cc:1428] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:29:32.958791 10220 heartbeater.cc:503] Master 127.21.81.254:38427 was elected leader, sending a full tablet report...
I20260809 19:29:32.958979 10243 tablet_bootstrap.cc:492] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6: Bootstrap starting.
I20260809 19:29:32.959056 10239 raft_consensus.cc:348] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08536806fa94af2bb41eb785e5422f6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45371 } } peers { permanent_uuid: "174eab7bb1544d20b7558eb70253152e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37255 } } peers { permanent_uuid: "dc3fdba3ca934282b221ea627a7f6b50" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 35839 } }
I20260809 19:29:32.959227 10239 raft_consensus.cc:374] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:32.959277 10239 raft_consensus.cc:729] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3fdba3ca934282b221ea627a7f6b50, State: Initialized, Role: FOLLOWER
I20260809 19:29:32.959434 10239 consensus_queue.cc:260] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08536806fa94af2bb41eb785e5422f6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45371 } } peers { permanent_uuid: "174eab7bb1544d20b7558eb70253152e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37255 } } peers { permanent_uuid: "dc3fdba3ca934282b221ea627a7f6b50" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 35839 } }
I20260809 19:29:32.959677 10243 tablet_bootstrap.cc:654] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:32.959717 10239 ts_tablet_manager.cc:1428] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:29:32.959856 9937 heartbeater.cc:503] Master 127.21.81.254:38427 was elected leader, sending a full tablet report...
I20260809 19:29:32.960278 10243 log.cc:824] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:32.961038 10243 tablet_bootstrap.cc:492] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6: No bootstrap required, opened a new log
I20260809 19:29:32.961119 10243 ts_tablet_manager.cc:1397] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:29:32.962677 10243 raft_consensus.cc:348] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08536806fa94af2bb41eb785e5422f6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45371 } } peers { permanent_uuid: "174eab7bb1544d20b7558eb70253152e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37255 } } peers { permanent_uuid: "dc3fdba3ca934282b221ea627a7f6b50" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 35839 } }
I20260809 19:29:32.962816 10243 raft_consensus.cc:374] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:32.962862 10243 raft_consensus.cc:729] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e08536806fa94af2bb41eb785e5422f6, State: Initialized, Role: FOLLOWER
I20260809 19:29:32.962989 10243 consensus_queue.cc:260] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08536806fa94af2bb41eb785e5422f6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45371 } } peers { permanent_uuid: "174eab7bb1544d20b7558eb70253152e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37255 } } peers { permanent_uuid: "dc3fdba3ca934282b221ea627a7f6b50" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 35839 } }
I20260809 19:29:32.963234 10243 ts_tablet_manager.cc:1428] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:29:32.963270 10077 heartbeater.cc:503] Master 127.21.81.254:38427 was elected leader, sending a full tablet report...
W20260809 19:29:33.004537 10078 tablet.cc:2369] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:29:33.004906 10245 raft_consensus.cc:482] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:29:33.005012 10245 raft_consensus.cc:504] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08536806fa94af2bb41eb785e5422f6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45371 } } peers { permanent_uuid: "174eab7bb1544d20b7558eb70253152e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37255 } } peers { permanent_uuid: "dc3fdba3ca934282b221ea627a7f6b50" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 35839 } }
I20260809 19:29:33.005313 10245 leader_election.cc:288] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e08536806fa94af2bb41eb785e5422f6 (127.21.81.194:45371), 174eab7bb1544d20b7558eb70253152e (127.21.81.195:37255)
I20260809 19:29:33.009267 10235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.005618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:43089 (local address 127.21.81.195:37255)
0809 19:29:33.005676 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:33.005679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:33.005863 (+ 184us) server_negotiation.cc:407] Connection header received
0809 19:29:33.005923 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:33.005924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:33.005933 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:33.005991 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:33.006136 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.006827 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.008343 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.008911 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.008987 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.008996 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:33.009161 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 19:29:33.009202 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 19:29:33.009266 10237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.005691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:33507 (local address 127.21.81.194:45371)
0809 19:29:33.005744 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:33.005746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:33.005755 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:29:33.005784 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:33.005785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:33.005795 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:33.005871 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:33.006064 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.006828 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.008343 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.008908 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.008988 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.008996 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:33.009160 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 19:29:33.009196 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:29:33.009280 9939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.005505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:45371 (local address 127.21.81.193:33507)
0809 19:29:33.005567 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:33.005597 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:33.005651 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:33.005889 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:33.005890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:33.005892 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:33.005982 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.005983 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.006911 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.006912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.008242 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.008244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.008994 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.008995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.009077 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.009079 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:29:33.009136 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 19:29:33.009180 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":84}
I20260809 19:29:33.009287 10248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.005506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:37255 (local address 127.21.81.193:43089)
0809 19:29:33.005794 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:33.005801 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:33.005869 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:33.006000 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:33.006001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:33.006002 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:33.006061 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.006062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.006911 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.006912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.008238 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.008241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.008994 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.008995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.009075 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.009079 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:29:33.009136 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 19:29:33.009180 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":173,"thread_start_us":136,"threads_started":1}
I20260809 19:29:33.009866 10172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f118f9bf51c42ec87dd47db938b3ee0" candidate_uuid: "dc3fdba3ca934282b221ea627a7f6b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "174eab7bb1544d20b7558eb70253152e" is_pre_election: true
I20260809 19:29:33.009881 10029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f118f9bf51c42ec87dd47db938b3ee0" candidate_uuid: "dc3fdba3ca934282b221ea627a7f6b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e08536806fa94af2bb41eb785e5422f6" is_pre_election: true
I20260809 19:29:33.009991 10172 raft_consensus.cc:2454] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc3fdba3ca934282b221ea627a7f6b50 in term 0.
I20260809 19:29:33.010057 10029 raft_consensus.cc:2454] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc3fdba3ca934282b221ea627a7f6b50 in term 0.
I20260809 19:29:33.010257 9820 leader_election.cc:302] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 174eab7bb1544d20b7558eb70253152e, dc3fdba3ca934282b221ea627a7f6b50; no voters:
I20260809 19:29:33.010386 10245 raft_consensus.cc:2781] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:29:33.010444 10245 raft_consensus.cc:482] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:29:33.010504 10245 raft_consensus.cc:3037] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:33.011472 10245 raft_consensus.cc:504] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08536806fa94af2bb41eb785e5422f6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45371 } } peers { permanent_uuid: "174eab7bb1544d20b7558eb70253152e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37255 } } peers { permanent_uuid: "dc3fdba3ca934282b221ea627a7f6b50" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 35839 } }
I20260809 19:29:33.011622 10245 leader_election.cc:288] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [CANDIDATE]: Term 1 election: Requested vote from peers e08536806fa94af2bb41eb785e5422f6 (127.21.81.194:45371), 174eab7bb1544d20b7558eb70253152e (127.21.81.195:37255)
I20260809 19:29:33.011752 10029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f118f9bf51c42ec87dd47db938b3ee0" candidate_uuid: "dc3fdba3ca934282b221ea627a7f6b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e08536806fa94af2bb41eb785e5422f6"
I20260809 19:29:33.011788 10172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f118f9bf51c42ec87dd47db938b3ee0" candidate_uuid: "dc3fdba3ca934282b221ea627a7f6b50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "174eab7bb1544d20b7558eb70253152e"
I20260809 19:29:33.011829 10029 raft_consensus.cc:3037] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:33.011878 10172 raft_consensus.cc:3037] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:33.012543 10029 raft_consensus.cc:2454] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc3fdba3ca934282b221ea627a7f6b50 in term 1.
I20260809 19:29:33.012717 9822 leader_election.cc:302] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc3fdba3ca934282b221ea627a7f6b50, e08536806fa94af2bb41eb785e5422f6; no voters:
I20260809 19:29:33.012758 10172 raft_consensus.cc:2454] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc3fdba3ca934282b221ea627a7f6b50 in term 1.
I20260809 19:29:33.012826 10245 raft_consensus.cc:2781] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:33.013023 10245 raft_consensus.cc:686] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [term 1 LEADER]: Becoming Leader. State: Replica: dc3fdba3ca934282b221ea627a7f6b50, State: Running, Role: LEADER
I20260809 19:29:33.013123 10245 consensus_queue.cc:237] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08536806fa94af2bb41eb785e5422f6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45371 } } peers { permanent_uuid: "174eab7bb1544d20b7558eb70253152e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37255 } } peers { permanent_uuid: "dc3fdba3ca934282b221ea627a7f6b50" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 35839 } }
I20260809 19:29:33.014009 9737 catalog_manager.cc:5495] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc3fdba3ca934282b221ea627a7f6b50 (127.21.81.193). New cstate: current_term: 1 leader_uuid: "dc3fdba3ca934282b221ea627a7f6b50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e08536806fa94af2bb41eb785e5422f6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "174eab7bb1544d20b7558eb70253152e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc3fdba3ca934282b221ea627a7f6b50" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 35839 } health_report { overall_health: HEALTHY } } }
I20260809 19:29:33.022903 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.020497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35636 (local address 127.21.81.193:35839)
0809 19:29:33.020542 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:33.020544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:33.020553 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:29:33.020636 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:33.020637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:33.020655 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:33.020734 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:29:33.020886 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.021324 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.022123 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.022324 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.022406 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.022585 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:29:33.022679 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:29:33.022763 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:33.022788 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 19:29:33.022808 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:29:33.024520 10172 raft_consensus.cc:1261] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e [term 1 FOLLOWER]: Refusing update from remote peer dc3fdba3ca934282b221ea627a7f6b50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:29:33.024824 10029 raft_consensus.cc:1261] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6 [term 1 FOLLOWER]: Refusing update from remote peer dc3fdba3ca934282b221ea627a7f6b50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:29:33.024856 10245 consensus_queue.cc:1035] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "174eab7bb1544d20b7558eb70253152e" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 37255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:29:33.025094 10245 consensus_queue.cc:1035] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e08536806fa94af2bb41eb785e5422f6" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 45371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:29:33.028505 10253 mvcc.cc:205] Tried to move back new op lower bound from 7316700254305976320 to 7316700254262382592. Current Snapshot: MvccSnapshot[applied={T|T < 7316700254305976320}]
I20260809 19:29:33.030092 9737 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:29:33.030207 9737 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:29:33.031482 9737 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d1dd5d2b7734c64a02c0f3a1a227d76: Sending DeleteTablet for 3 replicas of tablet 6f118f9bf51c42ec87dd47db938b3ee0
I20260809 19:29:33.031868 10008 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f118f9bf51c42ec87dd47db938b3ee0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:33 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:36920
I20260809 19:29:33.032207 10263 tablet_replica.cc:330] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6: stopping tablet replica
I20260809 19:29:33.032327 10263 raft_consensus.cc:2229] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:29:33.032428 10263 raft_consensus.cc:2258] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:29:33.031884 9869 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f118f9bf51c42ec87dd47db938b3ee0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:33 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:35628
I20260809 19:29:33.032651 10264 tablet_replica.cc:330] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50: stopping tablet replica
I20260809 19:29:33.032806 10264 raft_consensus.cc:2229] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:29:33.032871 10149 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f118f9bf51c42ec87dd47db938b3ee0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:33 UTC) from {username='kudu', principal='kudu/127.21.81.254@KRBTEST.COM'} at 127.0.0.1:39396
I20260809 19:29:33.032898 10263 ts_tablet_manager.cc:1905] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:29:33.033007 10264 raft_consensus.cc:2258] T 6f118f9bf51c42ec87dd47db938b3ee0 P dc3fdba3ca934282b221ea627a7f6b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:29:33.033368 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 9803
I20260809 19:29:33.036242 10263 ts_tablet_manager.cc:1918] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:29:33.036331 10263 log.cc:1196] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/6f118f9bf51c42ec87dd47db938b3ee0
I20260809 19:29:33.036576 10263 ts_tablet_manager.cc:1939] T 6f118f9bf51c42ec87dd47db938b3ee0 P e08536806fa94af2bb41eb785e5422f6: Deleting consensus metadata
I20260809 19:29:33.036984 9724 catalog_manager.cc:4841] TS e08536806fa94af2bb41eb785e5422f6 (127.21.81.194:45371): tablet 6f118f9bf51c42ec87dd47db938b3ee0 (table test-table [id=247aee7faba84e61bc5814a42283a390]) successfully deleted
I20260809 19:29:33.037693 10265 tablet_replica.cc:330] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e: stopping tablet replica
I20260809 19:29:33.037810 10265 raft_consensus.cc:2229] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:29:33.037928 10265 raft_consensus.cc:2258] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:29:33.038329 10265 ts_tablet_manager.cc:1905] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:29:33.039543 10265 ts_tablet_manager.cc:1918] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:29:33.039635 10265 log.cc:1196] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/6f118f9bf51c42ec87dd47db938b3ee0
I20260809 19:29:33.039886 10265 ts_tablet_manager.cc:1939] T 6f118f9bf51c42ec87dd47db938b3ee0 P 174eab7bb1544d20b7558eb70253152e: Deleting consensus metadata
I20260809 19:29:33.040306 9721 catalog_manager.cc:4841] TS 174eab7bb1544d20b7558eb70253152e (127.21.81.195:37255): tablet 6f118f9bf51c42ec87dd47db938b3ee0 (table test-table [id=247aee7faba84e61bc5814a42283a390]) successfully deleted
W20260809 19:29:33.042188 9720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:35839 (error 108)
I20260809 19:29:33.042697 10231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.042563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:35839 (local address 127.0.0.1:35652)
0809 19:29:33.042624 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:33.042668 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:35839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 19:29:33.042724 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 9942
W20260809 19:29:33.042834 9720 catalog_manager.cc:4584] TS dc3fdba3ca934282b221ea627a7f6b50 (127.21.81.193:35839): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f118f9bf51c42ec87dd47db938b3ee0: Network error: Client connection negotiation failed: client connection to 127.21.81.193:35839: connect: Connection refused (error 111)
I20260809 19:29:33.048687 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 10082
I20260809 19:29:33.054644 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 9704
2026-08-09T19:29:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1948 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T19:29:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:33Z Disabled control of system clock
I20260809 19:29:33.085445 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39423
--webserver_interface=127.21.81.254
--webserver_port=0
--builtin_ntp_servers=127.21.81.212:38133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:39423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:29:33.170253 10270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:33.170482 10270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:33.170502 10270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:33.171710 10270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:33.171756 10270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:33.171770 10270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:33.171782 10270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:33.171835 10270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:33.173183 10270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:38133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:39423
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:33.173449 10270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:33.173784 10270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:33.178761 10278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:33.178807 10276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:33.178776 10275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:33.178850 10270 server_base.cc:1034] running on GCE node
I20260809 19:29:33.179236 10270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:33.179526 10270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:33.180694 10270 hybrid_clock.cc:648] HybridClock initialized: now 1786303773180668 us; error 60 us; skew 500 ppm
I20260809 19:29:33.181996 10270 webserver.cc:460] Webserver started at http://127.21.81.254:42649/ using document root <none> and password file <none>
I20260809 19:29:33.182250 10270 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:33.182320 10270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:33.182471 10270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:33.183543 10270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "cff458512632477892d542ad96a34667"
format_stamp: "Formatted at 2026-08-09 19:29:33 on dist-test-slave-njxd"
server_key: "9778dea4637356cb3cf7220a1405fbf2"
server_key_iv: "287e685e84f016cb0a7590e5d902ca78"
server_key_version: "encryptionkey@0"
I20260809 19:29:33.183900 10270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "cff458512632477892d542ad96a34667"
format_stamp: "Formatted at 2026-08-09 19:29:33 on dist-test-slave-njxd"
server_key: "9778dea4637356cb3cf7220a1405fbf2"
server_key_iv: "287e685e84f016cb0a7590e5d902ca78"
server_key_version: "encryptionkey@0"
I20260809 19:29:33.185175 10270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:29:33.185938 10284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:33.186092 10270 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:33.186208 10270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "cff458512632477892d542ad96a34667"
format_stamp: "Formatted at 2026-08-09 19:29:33 on dist-test-slave-njxd"
server_key: "9778dea4637356cb3cf7220a1405fbf2"
server_key_iv: "287e685e84f016cb0a7590e5d902ca78"
server_key_version: "encryptionkey@0"
I20260809 19:29:33.186280 10270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:33.211619 10270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:33.211941 10270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:33.212101 10270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:33.216262 10270 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:39423
I20260809 19:29:33.216300 10341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:39423 every 8 connection(s)
I20260809 19:29:33.216643 10270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:33.217721 10342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:33.220032 10342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667: Bootstrap starting.
I20260809 19:29:33.220441 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 10270
I20260809 19:29:33.220564 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:33.220747 10342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:33.220772 21831 external_mini_cluster.cc:1428] Setting key bd52f48e49597ce116dd08203e2fd1d8
I20260809 19:29:33.221047 10342 log.cc:824] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:33.222023 10342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667: No bootstrap required, opened a new log
I20260809 19:29:33.224180 10342 raft_consensus.cc:348] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff458512632477892d542ad96a34667" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39423 } }
I20260809 19:29:33.224352 10342 raft_consensus.cc:374] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:33.224385 10342 raft_consensus.cc:729] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cff458512632477892d542ad96a34667, State: Initialized, Role: FOLLOWER
I20260809 19:29:33.224499 10342 consensus_queue.cc:260] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff458512632477892d542ad96a34667" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39423 } }
I20260809 19:29:33.224575 10342 raft_consensus.cc:388] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:33.224623 10342 raft_consensus.cc:482] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:33.224670 10342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:33.225260 10342 raft_consensus.cc:504] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff458512632477892d542ad96a34667" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39423 } }
I20260809 19:29:33.225375 10342 leader_election.cc:302] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cff458512632477892d542ad96a34667; no voters:
I20260809 19:29:33.225610 10342 leader_election.cc:288] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:33.225639 10348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:33.225787 10348 raft_consensus.cc:686] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [term 1 LEADER]: Becoming Leader. State: Replica: cff458512632477892d542ad96a34667, State: Running, Role: LEADER
I20260809 19:29:33.225922 10342 sys_catalog.cc:564] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:33.225924 10348 consensus_queue.cc:237] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff458512632477892d542ad96a34667" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39423 } }
I20260809 19:29:33.226711 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.221275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34872 (local address 127.21.81.254:39423)
0809 19:29:33.221495 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:33.221505 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:33.221542 (+ 37us) server_negotiation.cc:407] Connection header received
0809 19:29:33.222087 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:33.222096 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:33.222178 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:33.222294 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:33.222582 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.223452 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.224675 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.225116 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.225208 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.225996 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:33.226001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:33.226001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:33.226205 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:33.226239 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:33.226249 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:33.226298 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:33.226410 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 19:29:33.226490 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260809 19:29:33.226732 10349 sys_catalog.cc:455] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cff458512632477892d542ad96a34667. Latest consensus state: current_term: 1 leader_uuid: "cff458512632477892d542ad96a34667" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff458512632477892d542ad96a34667" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39423 } } }
I20260809 19:29:33.227061 10349 sys_catalog.cc:458] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:33.226743 10350 sys_catalog.cc:455] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cff458512632477892d542ad96a34667" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cff458512632477892d542ad96a34667" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39423 } } }
I20260809 19:29:33.227241 10350 sys_catalog.cc:458] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667 [sys.catalog]: This master's current role is: LEADER
W20260809 19:29:33.227829 10363 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:29:33.227887 10363 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:29:33.228204 10362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:33.228348 10362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:33.229837 10362 catalog_manager.cc:1347] Generated new cluster ID: 8853a34fefa84e94975de2e3aaa85ae0
I20260809 19:29:33.229888 10362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:33.239594 10362 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:33.240118 10362 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:33.247001 10362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667: Generated new TSK 0
I20260809 19:29:33.247200 10362 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:33.252044 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.193:0
--local_ip_for_outbound_sockets=127.21.81.193
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39423
--builtin_ntp_servers=127.21.81.212:38133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:29:33.335515 10367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:33.335716 10367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:33.335738 10367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:33.336898 10367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:33.336937 10367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:33.336963 10367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.193
I20260809 19:29:33.338326 10367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:38133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.81.193
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.193
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:33.338562 10367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:33.338886 10367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:33.344645 10374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:33.344789 10367 server_base.cc:1034] running on GCE node
W20260809 19:29:33.344702 10373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:33.344851 10376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:33.345062 10367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:33.345301 10367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:33.346467 10367 hybrid_clock.cc:648] HybridClock initialized: now 1786303773346447 us; error 39 us; skew 500 ppm
I20260809 19:29:33.347888 10367 webserver.cc:460] Webserver started at http://127.21.81.193:33969/ using document root <none> and password file <none>
I20260809 19:29:33.348146 10367 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:33.348214 10367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:33.348377 10367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:33.349457 10367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/data/instance:
uuid: "a32b443d55674fe09094b5272e2b0337"
format_stamp: "Formatted at 2026-08-09 19:29:33 on dist-test-slave-njxd"
server_key: "6b6d6448d7c294f730d9fc21da6a8524"
server_key_iv: "d23e93b17ef9d140e1a9f4ab3d205e5f"
server_key_version: "encryptionkey@0"
I20260809 19:29:33.349889 10367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance:
uuid: "a32b443d55674fe09094b5272e2b0337"
format_stamp: "Formatted at 2026-08-09 19:29:33 on dist-test-slave-njxd"
server_key: "6b6d6448d7c294f730d9fc21da6a8524"
server_key_iv: "d23e93b17ef9d140e1a9f4ab3d205e5f"
server_key_version: "encryptionkey@0"
I20260809 19:29:33.351296 10367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:29:33.352085 10382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:33.352334 10367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:29:33.352447 10367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/wal
uuid: "a32b443d55674fe09094b5272e2b0337"
format_stamp: "Formatted at 2026-08-09 19:29:33 on dist-test-slave-njxd"
server_key: "6b6d6448d7c294f730d9fc21da6a8524"
server_key_iv: "d23e93b17ef9d140e1a9f4ab3d205e5f"
server_key_version: "encryptionkey@0"
I20260809 19:29:33.352523 10367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:33.360090 10367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:33.360384 10367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:33.360532 10367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:33.360762 10367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:33.361086 10367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:33.361140 10367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:33.361181 10367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:33.361209 10367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:33.367370 10367 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.193:37351
I20260809 19:29:33.367417 10500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.193:37351 every 8 connection(s)
I20260809 19:29:33.367748 10367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/data/info.pb
I20260809 19:29:33.372834 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.368465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:53443 (local address 127.21.81.254:39423)
0809 19:29:33.368532 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:33.368534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:33.368786 (+ 252us) server_negotiation.cc:407] Connection header received
0809 19:29:33.369278 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:33.369279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:33.369289 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:33.369347 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:33.370092 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.370583 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.371535 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.371713 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.371769 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.372637 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:33.372638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:33.372638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:33.372643 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:33.372652 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:33.372655 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:33.372691 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:33.372756 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 19:29:33.372775 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:29:33.373088 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.368440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39423 (local address 127.21.81.193:53443)
0809 19:29:33.368655 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:33.368697 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:33.369186 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:33.369419 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:33.369422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:33.369430 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:33.370010 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.370016 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.370677 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.370678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.371458 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.371459 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.371793 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.371794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.371901 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.372513 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:29:33.372513 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:33.372542 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:29:33.372566 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:33.372693 (+ 127us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:33.372696 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:33.372698 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:29:33.372743 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:29:33.372825 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
I20260809 19:29:33.373672 10501 heartbeater.cc:347] Connected to a master server at 127.21.81.254:39423
I20260809 19:29:33.373786 10501 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:33.373984 10501 heartbeater.cc:511] Master 127.21.81.254:39423 requested a full tablet report, sending...
I20260809 19:29:33.374492 10303 ts_manager.cc:194] Registered new tserver with Master: a32b443d55674fe09094b5272e2b0337 (127.21.81.193:37351)
I20260809 19:29:33.375079 10303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.81.193:53443
I20260809 19:29:33.377760 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 10367
I20260809 19:29:33.377884 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-0/wal/instance
I20260809 19:29:33.378121 21831 external_mini_cluster.cc:1428] Setting key 41474e62fde8bedd1af3d60bf040af0e
I20260809 19:29:33.378964 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.194:0
--local_ip_for_outbound_sockets=127.21.81.194
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39423
--builtin_ntp_servers=127.21.81.212:38133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:29:33.462767 10504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:33.463011 10504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:33.463052 10504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:33.464242 10504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:33.464318 10504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:33.464365 10504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.194
I20260809 19:29:33.465881 10504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:38133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.81.194
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.194
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:33.466150 10504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:33.466473 10504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:33.471601 10512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:33.471601 10509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:33.471729 10504 server_base.cc:1034] running on GCE node
W20260809 19:29:33.471872 10510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:33.472143 10504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:33.472434 10504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:33.473628 10504 hybrid_clock.cc:648] HybridClock initialized: now 1786303773473607 us; error 39 us; skew 500 ppm
I20260809 19:29:33.474978 10504 webserver.cc:460] Webserver started at http://127.21.81.194:37537/ using document root <none> and password file <none>
I20260809 19:29:33.475194 10504 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:33.475237 10504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:33.475337 10504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:33.476411 10504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/data/instance:
uuid: "c7b721dce4034e5186b3c9b8dae764b5"
format_stamp: "Formatted at 2026-08-09 19:29:33 on dist-test-slave-njxd"
server_key: "25c5beb3a39ae379748616284ab98f72"
server_key_iv: "6bb5f837383d03d55fc25351a7a1413c"
server_key_version: "encryptionkey@0"
I20260809 19:29:33.476724 10504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance:
uuid: "c7b721dce4034e5186b3c9b8dae764b5"
format_stamp: "Formatted at 2026-08-09 19:29:33 on dist-test-slave-njxd"
server_key: "25c5beb3a39ae379748616284ab98f72"
server_key_iv: "6bb5f837383d03d55fc25351a7a1413c"
server_key_version: "encryptionkey@0"
I20260809 19:29:33.478101 10504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:29:33.478773 10518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:33.478947 10504 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:33.479045 10504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/wal
uuid: "c7b721dce4034e5186b3c9b8dae764b5"
format_stamp: "Formatted at 2026-08-09 19:29:33 on dist-test-slave-njxd"
server_key: "25c5beb3a39ae379748616284ab98f72"
server_key_iv: "6bb5f837383d03d55fc25351a7a1413c"
server_key_version: "encryptionkey@0"
I20260809 19:29:33.479105 10504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:33.495033 10504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:33.495309 10504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:33.495415 10504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:33.495602 10504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:33.495968 10504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:33.496007 10504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:33.496053 10504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:33.496081 10504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:33.502215 10504 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.194:35607
I20260809 19:29:33.502303 10636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.194:35607 every 8 connection(s)
I20260809 19:29:33.502636 10504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/data/info.pb
I20260809 19:29:33.504698 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 10504
I20260809 19:29:33.504791 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/wal/instance
I20260809 19:29:33.504972 21831 external_mini_cluster.cc:1428] Setting key 0fef949989b0c9535eac3c026093a558
I20260809 19:29:33.506013 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.81.195:0
--local_ip_for_outbound_sockets=127.21.81.195
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39423
--builtin_ntp_servers=127.21.81.212:38133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 19:29:33.508541 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.503419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.194:60853 (local address 127.21.81.254:39423)
0809 19:29:33.503488 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:33.503490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:33.503719 (+ 229us) server_negotiation.cc:407] Connection header received
0809 19:29:33.504253 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:33.504255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:33.504269 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:33.504330 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:33.505138 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.505861 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.506947 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.507150 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.507226 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.508279 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:33.508280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:33.508280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:33.508285 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:33.508299 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:33.508303 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:33.508352 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:33.508460 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 19:29:33.508481 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:29:33.508828 10639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.503411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39423 (local address 127.21.81.194:60853)
0809 19:29:33.503590 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:33.503625 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:33.504141 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:33.504405 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:33.504409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:33.504417 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:33.505030 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.505039 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.505957 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.505958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.506851 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.506853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.507249 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.507250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.507389 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.508086 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:29:33.508087 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:33.508130 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:29:33.508161 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:33.508372 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:33.508378 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:33.508380 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:29:33.508456 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 19:29:33.508575 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260809 19:29:33.509281 10637 heartbeater.cc:347] Connected to a master server at 127.21.81.254:39423
I20260809 19:29:33.509382 10637 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:33.509608 10637 heartbeater.cc:511] Master 127.21.81.254:39423 requested a full tablet report, sending...
I20260809 19:29:33.509986 10303 ts_manager.cc:194] Registered new tserver with Master: c7b721dce4034e5186b3c9b8dae764b5 (127.21.81.194:35607)
I20260809 19:29:33.510375 10303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.81.194:60853
W20260809 19:29:33.590310 10640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:33.590531 10640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:33.590569 10640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:33.591789 10640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:33.591845 10640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:33.591886 10640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.81.195
I20260809 19:29:33.593245 10640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:38133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.81.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.81.195
--webserver_port=0
--tserver_master_addrs=127.21.81.254:39423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.81.195
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:33.593508 10640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:33.593850 10640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:33.598999 10648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:33.598964 10647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:33.599012 10640 server_base.cc:1034] running on GCE node
W20260809 19:29:33.598981 10650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:33.599478 10640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:33.599715 10640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:33.600879 10640 hybrid_clock.cc:648] HybridClock initialized: now 1786303773600867 us; error 30 us; skew 500 ppm
I20260809 19:29:33.602195 10640 webserver.cc:460] Webserver started at http://127.21.81.195:43537/ using document root <none> and password file <none>
I20260809 19:29:33.602418 10640 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:33.602458 10640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:33.602542 10640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:33.603521 10640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/data/instance:
uuid: "8b7f85e7cdfa473897d838e26a7df5c4"
format_stamp: "Formatted at 2026-08-09 19:29:33 on dist-test-slave-njxd"
server_key: "119c030b46d18d34de941322b958a251"
server_key_iv: "d4c0716b0ce34452f957a8a352931e37"
server_key_version: "encryptionkey@0"
I20260809 19:29:33.603816 10640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance:
uuid: "8b7f85e7cdfa473897d838e26a7df5c4"
format_stamp: "Formatted at 2026-08-09 19:29:33 on dist-test-slave-njxd"
server_key: "119c030b46d18d34de941322b958a251"
server_key_iv: "d4c0716b0ce34452f957a8a352931e37"
server_key_version: "encryptionkey@0"
I20260809 19:29:33.605033 10640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:29:33.605779 10656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:33.605949 10640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:29:33.606036 10640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/wal
uuid: "8b7f85e7cdfa473897d838e26a7df5c4"
format_stamp: "Formatted at 2026-08-09 19:29:33 on dist-test-slave-njxd"
server_key: "119c030b46d18d34de941322b958a251"
server_key_iv: "d4c0716b0ce34452f957a8a352931e37"
server_key_version: "encryptionkey@0"
I20260809 19:29:33.606089 10640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:33.620666 10640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:33.620931 10640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:33.621034 10640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:33.621233 10640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:29:33.621634 10640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:29:33.621673 10640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:33.621722 10640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:29:33.621752 10640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:33.627792 10640 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.195:42435
I20260809 19:29:33.627871 10776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.195:42435 every 8 connection(s)
I20260809 19:29:33.628213 10640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/data/info.pb
I20260809 19:29:33.632117 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 10640
I20260809 19:29:33.632251 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/wal/instance
I20260809 19:29:33.632438 21831 external_mini_cluster.cc:1428] Setting key 3bb629216cfba71ef4be39089372887b
I20260809 19:29:33.633562 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.628943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.195:50469 (local address 127.21.81.254:39423)
0809 19:29:33.629012 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:33.629014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:33.629235 (+ 221us) server_negotiation.cc:407] Connection header received
0809 19:29:33.629827 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:33.629828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:33.629841 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:33.629900 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:33.630572 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.631081 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.632115 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.632416 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.632478 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.633335 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:33.633336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:33.633336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:33.633341 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:33.633356 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:33.633360 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:33.633414 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:33.633484 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 19:29:33.633504 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:29:33.633816 10779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.628924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:39423 (local address 127.21.81.195:50469)
0809 19:29:33.629119 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:33.629152 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:33.629721 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:33.629971 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:33.629974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:33.629982 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:33.630468 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.630472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.631172 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.631173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.632025 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.632028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.632513 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.632514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.632661 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.633216 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:29:33.633217 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:33.633250 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:29:33.633272 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:33.633412 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:33.633415 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:33.633418 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:29:33.633487 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 19:29:33.633562 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260809 19:29:33.634269 10777 heartbeater.cc:347] Connected to a master server at 127.21.81.254:39423
I20260809 19:29:33.634349 10777 heartbeater.cc:464] Registering TS with master...
I20260809 19:29:33.634554 10777 heartbeater.cc:511] Master 127.21.81.254:39423 requested a full tablet report, sending...
I20260809 19:29:33.634951 10303 ts_manager.cc:194] Registered new tserver with Master: 8b7f85e7cdfa473897d838e26a7df5c4 (127.21.81.195:42435)
I20260809 19:29:33.635325 10303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.81.195:50469
I20260809 19:29:33.644281 21831 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:29:33.648409 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.645726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34888 (local address 127.21.81.254:39423)
0809 19:29:33.645826 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:33.645832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:33.645869 (+ 37us) server_negotiation.cc:407] Connection header received
0809 19:29:33.645924 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:33.645924 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:33.645934 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:33.645982 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:33.646126 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.646592 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.647299 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.647474 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.647523 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.648168 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:33.648168 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:33.648169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:33.648172 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:33.648183 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:33.648186 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:33.648224 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:33.648288 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 19:29:33.648305 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 19:29:33.649955 10303 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:29:33.651563 10303 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:29:33.658437 10787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.653885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:37351 (local address 127.0.0.1:60638)
0809 19:29:33.654377 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:33.654387 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:33.654440 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:33.654665 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:33.654666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:33.654669 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:33.654771 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.654773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.656111 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.656112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.657130 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.657133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.658132 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.658133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.658211 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.658226 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 19:29:33.658279 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 19:29:33.658326 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":18,"spinlock_wait_cycles":384,"thread_start_us":99,"threads_started":1}
I20260809 19:29:33.658449 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.654010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56920 (local address 127.21.81.194:35607)
0809 19:29:33.654312 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:33.654316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:33.654326 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:29:33.654465 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:33.654467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:33.654484 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:33.654577 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:33.654953 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.656070 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.657459 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.658032 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.658124 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.658219 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:33.658328 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 19:29:33.658365 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":98,"threads_started":1}
I20260809 19:29:33.658711 10786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.653885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:35607 (local address 127.0.0.1:56920)
0809 19:29:33.654207 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:33.654252 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:33.654321 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:33.654663 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:33.654666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:33.654667 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:33.654870 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.654872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.656136 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.656137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.657386 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.657389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.658128 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.658129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.658214 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.658228 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 19:29:33.658314 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 19:29:33.658359 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":43,"thread_start_us":80,"threads_started":1}
I20260809 19:29:33.659111 10789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.654010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.21.81.193:37351)
0809 19:29:33.654214 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:33.654218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:33.654450 (+ 232us) server_negotiation.cc:407] Connection header received
0809 19:29:33.654504 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:33.654507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:33.654524 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:33.654710 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:33.654837 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.656004 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.657225 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.658031 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.658407 (+ 376us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.658955 (+ 548us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:33.658983 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 19:29:33.659026 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":93,"threads_started":1}
I20260809 19:29:33.659631 10791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.654111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:42435 (local address 127.0.0.1:56302)
0809 19:29:33.654518 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:33.654526 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:33.654582 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:33.654840 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:33.654841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:33.654842 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:33.654872 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.654873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.657367 (+ 2494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.657368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.658423 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.658426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.659371 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.659372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.659439 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.659443 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:29:33.659523 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 19:29:33.659567 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":66,"threads_started":1}
I20260809 19:29:33.659657 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.654210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56302 (local address 127.21.81.195:42435)
0809 19:29:33.654381 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:33.654385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:33.654643 (+ 258us) server_negotiation.cc:407] Connection header received
0809 19:29:33.654689 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:33.654691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:33.654706 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:33.654775 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:33.656163 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.657282 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.658515 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.659277 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.659359 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.659458 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:33.659539 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 19:29:33.659577 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260809 19:29:33.659519 10569 tablet_service.cc:1467] Processing CreateTablet for tablet 6ad0c247f7c5420285907df9f741c998 (DEFAULT_TABLE table=test-table [id=25fdc350e29e446399b0e3f81e648ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:33.659961 10569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ad0c247f7c5420285907df9f741c998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:33.659893 10431 tablet_service.cc:1467] Processing CreateTablet for tablet 6ad0c247f7c5420285907df9f741c998 (DEFAULT_TABLE table=test-table [id=25fdc350e29e446399b0e3f81e648ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:33.660195 10431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ad0c247f7c5420285907df9f741c998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:33.660300 10706 tablet_service.cc:1467] Processing CreateTablet for tablet 6ad0c247f7c5420285907df9f741c998 (DEFAULT_TABLE table=test-table [id=25fdc350e29e446399b0e3f81e648ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:29:33.660604 10706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ad0c247f7c5420285907df9f741c998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:33.662802 10793 tablet_bootstrap.cc:492] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337: Bootstrap starting.
I20260809 19:29:33.663508 10793 tablet_bootstrap.cc:654] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:33.663498 10794 tablet_bootstrap.cc:492] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5: Bootstrap starting.
I20260809 19:29:33.663964 10793 log.cc:824] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:33.664203 10794 tablet_bootstrap.cc:654] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:33.664537 10794 log.cc:824] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:33.664836 10793 tablet_bootstrap.cc:492] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337: No bootstrap required, opened a new log
I20260809 19:29:33.664911 10793 ts_tablet_manager.cc:1397] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:29:33.664913 10795 tablet_bootstrap.cc:492] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4: Bootstrap starting.
I20260809 19:29:33.665269 10794 tablet_bootstrap.cc:492] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5: No bootstrap required, opened a new log
I20260809 19:29:33.665356 10794 ts_tablet_manager.cc:1397] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:29:33.665563 10795 tablet_bootstrap.cc:654] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:33.665962 10795 log.cc:824] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:33.667136 10793 raft_consensus.cc:348] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b443d55674fe09094b5272e2b0337" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37351 } } peers { permanent_uuid: "c7b721dce4034e5186b3c9b8dae764b5" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 35607 } } peers { permanent_uuid: "8b7f85e7cdfa473897d838e26a7df5c4" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42435 } }
I20260809 19:29:33.667361 10793 raft_consensus.cc:374] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:33.667415 10793 raft_consensus.cc:729] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a32b443d55674fe09094b5272e2b0337, State: Initialized, Role: FOLLOWER
I20260809 19:29:33.667543 10793 consensus_queue.cc:260] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b443d55674fe09094b5272e2b0337" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37351 } } peers { permanent_uuid: "c7b721dce4034e5186b3c9b8dae764b5" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 35607 } } peers { permanent_uuid: "8b7f85e7cdfa473897d838e26a7df5c4" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42435 } }
I20260809 19:29:33.667520 10794 raft_consensus.cc:348] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b443d55674fe09094b5272e2b0337" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37351 } } peers { permanent_uuid: "c7b721dce4034e5186b3c9b8dae764b5" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 35607 } } peers { permanent_uuid: "8b7f85e7cdfa473897d838e26a7df5c4" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42435 } }
I20260809 19:29:33.667693 10794 raft_consensus.cc:374] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:33.667730 10794 raft_consensus.cc:729] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b721dce4034e5186b3c9b8dae764b5, State: Initialized, Role: FOLLOWER
I20260809 19:29:33.667827 10793 ts_tablet_manager.cc:1428] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:29:33.667841 10794 consensus_queue.cc:260] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b443d55674fe09094b5272e2b0337" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37351 } } peers { permanent_uuid: "c7b721dce4034e5186b3c9b8dae764b5" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 35607 } } peers { permanent_uuid: "8b7f85e7cdfa473897d838e26a7df5c4" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42435 } }
I20260809 19:29:33.667889 10501 heartbeater.cc:503] Master 127.21.81.254:39423 was elected leader, sending a full tablet report...
I20260809 19:29:33.668099 10794 ts_tablet_manager.cc:1428] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:29:33.668116 10637 heartbeater.cc:503] Master 127.21.81.254:39423 was elected leader, sending a full tablet report...
I20260809 19:29:33.668812 10795 tablet_bootstrap.cc:492] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4: No bootstrap required, opened a new log
I20260809 19:29:33.669005 10795 ts_tablet_manager.cc:1397] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 19:29:33.670934 10795 raft_consensus.cc:348] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b443d55674fe09094b5272e2b0337" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37351 } } peers { permanent_uuid: "c7b721dce4034e5186b3c9b8dae764b5" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 35607 } } peers { permanent_uuid: "8b7f85e7cdfa473897d838e26a7df5c4" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42435 } }
I20260809 19:29:33.671051 10795 raft_consensus.cc:374] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:33.671072 10795 raft_consensus.cc:729] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b7f85e7cdfa473897d838e26a7df5c4, State: Initialized, Role: FOLLOWER
I20260809 19:29:33.671241 10795 consensus_queue.cc:260] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b443d55674fe09094b5272e2b0337" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37351 } } peers { permanent_uuid: "c7b721dce4034e5186b3c9b8dae764b5" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 35607 } } peers { permanent_uuid: "8b7f85e7cdfa473897d838e26a7df5c4" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42435 } }
I20260809 19:29:33.671535 10795 ts_tablet_manager.cc:1428] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:29:33.671648 10777 heartbeater.cc:503] Master 127.21.81.254:39423 was elected leader, sending a full tablet report...
I20260809 19:29:33.730865 10799 raft_consensus.cc:482] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:29:33.730996 10799 raft_consensus.cc:504] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b443d55674fe09094b5272e2b0337" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37351 } } peers { permanent_uuid: "c7b721dce4034e5186b3c9b8dae764b5" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 35607 } } peers { permanent_uuid: "8b7f85e7cdfa473897d838e26a7df5c4" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42435 } }
I20260809 19:29:33.731449 10799 leader_election.cc:288] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7b721dce4034e5186b3c9b8dae764b5 (127.21.81.194:35607), 8b7f85e7cdfa473897d838e26a7df5c4 (127.21.81.195:42435)
I20260809 19:29:33.734360 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.731706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:46575 (local address 127.21.81.194:35607)
0809 19:29:33.731758 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:33.731760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:33.731772 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:29:33.731878 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:33.731880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:33.731889 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:33.731976 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:33.732145 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.732631 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.733617 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.733993 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.734080 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.734088 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:33.734237 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 19:29:33.734274 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:29:33.734370 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.731647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.194:35607 (local address 127.21.81.193:46575)
0809 19:29:33.731709 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:33.731721 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:33.731765 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:33.731983 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:33.731984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:33.731985 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:33.732067 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.732069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.732725 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.732726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.733485 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.733487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.734079 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.734080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.734151 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.734162 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 19:29:33.734231 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 19:29:33.734271 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 19:29:33.734414 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.731754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.81.193:56529 (local address 127.21.81.195:42435)
0809 19:29:33.731829 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:33.731845 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:33.731950 (+ 105us) server_negotiation.cc:407] Connection header received
0809 19:29:33.732045 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:33.732046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:33.732053 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:33.732113 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:29:33.732275 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.732728 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.733677 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.734036 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.734121 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.734129 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:33.734297 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 19:29:33.734336 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 19:29:33.734475 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.731680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.195:42435 (local address 127.21.81.193:56529)
0809 19:29:33.731874 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:33.731884 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:33.731968 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:33.732117 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:33.732117 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:33.732119 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:29:33.732200 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.732202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.732815 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.732816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.733560 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:33.733562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.734122 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.734123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:33.734212 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.734215 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:29:33.734280 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 19:29:33.734319 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":108,"threads_started":1}
I20260809 19:29:33.734726 10589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ad0c247f7c5420285907df9f741c998" candidate_uuid: "a32b443d55674fe09094b5272e2b0337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b721dce4034e5186b3c9b8dae764b5" is_pre_election: true
I20260809 19:29:33.734901 10589 raft_consensus.cc:2454] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a32b443d55674fe09094b5272e2b0337 in term 0.
I20260809 19:29:33.734990 10726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ad0c247f7c5420285907df9f741c998" candidate_uuid: "a32b443d55674fe09094b5272e2b0337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b7f85e7cdfa473897d838e26a7df5c4" is_pre_election: true
I20260809 19:29:33.735172 10726 raft_consensus.cc:2454] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a32b443d55674fe09094b5272e2b0337 in term 0.
I20260809 19:29:33.735152 10383 leader_election.cc:302] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a32b443d55674fe09094b5272e2b0337, c7b721dce4034e5186b3c9b8dae764b5; no voters:
I20260809 19:29:33.735299 10799 raft_consensus.cc:2781] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:29:33.735360 10799 raft_consensus.cc:482] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:29:33.735390 10799 raft_consensus.cc:3037] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:33.736127 10799 raft_consensus.cc:504] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b443d55674fe09094b5272e2b0337" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37351 } } peers { permanent_uuid: "c7b721dce4034e5186b3c9b8dae764b5" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 35607 } } peers { permanent_uuid: "8b7f85e7cdfa473897d838e26a7df5c4" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42435 } }
I20260809 19:29:33.736285 10799 leader_election.cc:288] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [CANDIDATE]: Term 1 election: Requested vote from peers c7b721dce4034e5186b3c9b8dae764b5 (127.21.81.194:35607), 8b7f85e7cdfa473897d838e26a7df5c4 (127.21.81.195:42435)
I20260809 19:29:33.736419 10589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ad0c247f7c5420285907df9f741c998" candidate_uuid: "a32b443d55674fe09094b5272e2b0337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b721dce4034e5186b3c9b8dae764b5"
I20260809 19:29:33.736483 10726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ad0c247f7c5420285907df9f741c998" candidate_uuid: "a32b443d55674fe09094b5272e2b0337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b7f85e7cdfa473897d838e26a7df5c4"
I20260809 19:29:33.736515 10589 raft_consensus.cc:3037] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:33.736583 10726 raft_consensus.cc:3037] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:33.737313 10589 raft_consensus.cc:2454] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a32b443d55674fe09094b5272e2b0337 in term 1.
I20260809 19:29:33.737392 10726 raft_consensus.cc:2454] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a32b443d55674fe09094b5272e2b0337 in term 1.
I20260809 19:29:33.737519 10383 leader_election.cc:302] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a32b443d55674fe09094b5272e2b0337, c7b721dce4034e5186b3c9b8dae764b5; no voters:
I20260809 19:29:33.737658 10799 raft_consensus.cc:2781] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:33.737885 10799 raft_consensus.cc:686] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [term 1 LEADER]: Becoming Leader. State: Replica: a32b443d55674fe09094b5272e2b0337, State: Running, Role: LEADER
I20260809 19:29:33.738108 10799 consensus_queue.cc:237] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b443d55674fe09094b5272e2b0337" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37351 } } peers { permanent_uuid: "c7b721dce4034e5186b3c9b8dae764b5" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 35607 } } peers { permanent_uuid: "8b7f85e7cdfa473897d838e26a7df5c4" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42435 } }
I20260809 19:29:33.739023 10303 catalog_manager.cc:5495] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 reported cstate change: term changed from 0 to 1, leader changed from <none> to a32b443d55674fe09094b5272e2b0337 (127.21.81.193). New cstate: current_term: 1 leader_uuid: "a32b443d55674fe09094b5272e2b0337" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b443d55674fe09094b5272e2b0337" member_type: VOTER last_known_addr { host: "127.21.81.193" port: 37351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7b721dce4034e5186b3c9b8dae764b5" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 35607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b7f85e7cdfa473897d838e26a7df5c4" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42435 } health_report { overall_health: UNKNOWN } } }
I20260809 19:29:33.751567 10789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.749302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60650 (local address 127.21.81.193:37351)
0809 19:29:33.749368 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:33.749370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:33.749378 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:29:33.749409 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:33.749409 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:33.749427 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:33.749488 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:29:33.749664 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.750110 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.750835 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:33.751013 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:33.751062 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:33.751231 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:29:33.751334 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:29:33.751393 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:33.751429 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 19:29:33.751446 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 19:29:33.753207 10638 tablet.cc:2369] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:29:33.753429 10589 raft_consensus.cc:1261] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5 [term 1 FOLLOWER]: Refusing update from remote peer a32b443d55674fe09094b5272e2b0337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:29:33.753811 10726 raft_consensus.cc:1261] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4 [term 1 FOLLOWER]: Refusing update from remote peer a32b443d55674fe09094b5272e2b0337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:29:33.753775 10799 consensus_queue.cc:1035] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7b721dce4034e5186b3c9b8dae764b5" member_type: VOTER last_known_addr { host: "127.21.81.194" port: 35607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:29:33.754102 10799 consensus_queue.cc:1035] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b7f85e7cdfa473897d838e26a7df5c4" member_type: VOTER last_known_addr { host: "127.21.81.195" port: 42435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:29:33.756467 10807 mvcc.cc:205] Tried to move back new op lower bound from 7316700257291218944 to 7316700257231941632. Current Snapshot: MvccSnapshot[applied={T|T < 7316700257291218944}]
I20260809 19:29:33.756771 10809 mvcc.cc:205] Tried to move back new op lower bound from 7316700257291218944 to 7316700257231941632. Current Snapshot: MvccSnapshot[applied={T|T < 7316700257291218944}]
I20260809 19:29:33.759274 10301 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:29:33.759369 10301 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:29:33.760170 10301 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cff458512632477892d542ad96a34667: Sending DeleteTablet for 3 replicas of tablet 6ad0c247f7c5420285907df9f741c998
I20260809 19:29:33.760535 10431 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ad0c247f7c5420285907df9f741c998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:33 UTC) from {username='slave'} at 127.0.0.1:60638
I20260809 19:29:33.760566 10706 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ad0c247f7c5420285907df9f741c998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:33 UTC) from {username='slave'} at 127.0.0.1:56302
I20260809 19:29:33.760810 10818 tablet_replica.cc:330] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4: stopping tablet replica
I20260809 19:29:33.760970 10818 raft_consensus.cc:2229] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:29:33.760974 10569 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ad0c247f7c5420285907df9f741c998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:29:33 UTC) from {username='slave'} at 127.0.0.1:56920
I20260809 19:29:33.761181 10819 tablet_replica.cc:330] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5: stopping tablet replica
I20260809 19:29:33.761209 10817 tablet_replica.cc:330] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337: stopping tablet replica
I20260809 19:29:33.761238 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 10367
I20260809 19:29:33.761309 10817 raft_consensus.cc:2229] T 6ad0c247f7c5420285907df9f741c998 P a32b443d55674fe09094b5272e2b0337 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:29:33.761287 10819 raft_consensus.cc:2229] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:29:33.764674 10819 raft_consensus.cc:2258] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:29:33.765352 10819 ts_tablet_manager.cc:1905] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:29:33.766907 10818 raft_consensus.cc:2258] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:29:33.767428 10819 ts_tablet_manager.cc:1918] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:29:33.767459 10818 ts_tablet_manager.cc:1905] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:29:33.767518 10819 log.cc:1196] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-1/wal/wals/6ad0c247f7c5420285907df9f741c998
I20260809 19:29:33.767768 10819 ts_tablet_manager.cc:1939] T 6ad0c247f7c5420285907df9f741c998 P c7b721dce4034e5186b3c9b8dae764b5: Deleting consensus metadata
I20260809 19:29:33.768162 10285 catalog_manager.cc:4841] TS c7b721dce4034e5186b3c9b8dae764b5 (127.21.81.194:35607): tablet 6ad0c247f7c5420285907df9f741c998 (table test-table [id=25fdc350e29e446399b0e3f81e648ca4]) successfully deleted
I20260809 19:29:33.768762 10818 ts_tablet_manager.cc:1918] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:29:33.768886 10818 log.cc:1196] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4: Deleting WAL directory at /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303525464116-21831-0/minicluster-data/ts-2/wal/wals/6ad0c247f7c5420285907df9f741c998
I20260809 19:29:33.769152 10818 ts_tablet_manager.cc:1939] T 6ad0c247f7c5420285907df9f741c998 P 8b7f85e7cdfa473897d838e26a7df5c4: Deleting consensus metadata
I20260809 19:29:33.769536 10287 catalog_manager.cc:4841] TS 8b7f85e7cdfa473897d838e26a7df5c4 (127.21.81.195:42435): tablet 6ad0c247f7c5420285907df9f741c998 (table test-table [id=25fdc350e29e446399b0e3f81e648ca4]) successfully deleted
W20260809 19:29:33.772320 10287 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.81.193:37351 (error 108)
I20260809 19:29:33.772537 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 10504
I20260809 19:29:33.773653 10791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:33.772660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.193:37351 (local address 127.0.0.1:60662)
0809 19:29:33.773531 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:33.773608 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.193:37351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":847}
W20260809 19:29:33.773988 10287 catalog_manager.cc:4584] TS a32b443d55674fe09094b5272e2b0337 (127.21.81.193:37351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6ad0c247f7c5420285907df9f741c998: Network error: Client connection negotiation failed: client connection to 127.21.81.193:37351: connect: Connection refused (error 111)
I20260809 19:29:33.779475 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 10640
I20260809 19:29:33.786468 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 10270
2026-08-09T19:29:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (729 ms)
[----------] 6 tests from EncryptionPolicyTest (8551 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:33 dist-test-slave-njxd krb5kdc[10824](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:33 dist-test-slave-njxd krb5kdc[10824](info): set up 2 sockets
Aug 09 19:29:33 dist-test-slave-njxd krb5kdc[10824](info): commencing operation
krb5kdc: starting...
W20260809 19:29:34.845111 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:34 dist-test-slave-njxd krb5kdc[10824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:29:34.965745 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:35.049082 10838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:35.049335 10838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:35.049376 10838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:35.050670 10838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:35.050734 10838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:35.050758 10838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:35.050786 10838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:35.050817 10838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:35.052400 10838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32823
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:35.052695 10838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:35.052935 10838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:35.057785 10843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:35.057837 10844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:35.057854 10838 server_base.cc:1034] running on GCE node
W20260809 19:29:35.057838 10846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:35.058215 10838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:35.058573 10838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:35.059755 10838 hybrid_clock.cc:648] HybridClock initialized: now 1786303775059741 us; error 44 us; skew 500 ppm
Aug 09 19:29:35 dist-test-slave-njxd krb5kdc[10824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:35.062238 10838 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:29:35.062718 10838 webserver.cc:460] Webserver started at http://127.0.0.1:43387/ using document root <none> and password file <none>
I20260809 19:29:35.062958 10838 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:35.063004 10838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:35.063175 10838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:35.064213 10838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "d246e471456a498fa55b7034da709e04"
format_stamp: "Formatted at 2026-08-09 19:29:35 on dist-test-slave-njxd"
server_key: "1d13ca09930d8d26183e82d96f5c8760"
server_key_iv: "6ec79f6a87f696415176ba492368d425"
server_key_version: "encryptionkey@0"
I20260809 19:29:35.064596 10838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "d246e471456a498fa55b7034da709e04"
format_stamp: "Formatted at 2026-08-09 19:29:35 on dist-test-slave-njxd"
server_key: "1d13ca09930d8d26183e82d96f5c8760"
server_key_iv: "6ec79f6a87f696415176ba492368d425"
server_key_version: "encryptionkey@0"
I20260809 19:29:35.065982 10838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:29:35.066692 10853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:35.066851 10838 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:35.066959 10838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "d246e471456a498fa55b7034da709e04"
format_stamp: "Formatted at 2026-08-09 19:29:35 on dist-test-slave-njxd"
server_key: "1d13ca09930d8d26183e82d96f5c8760"
server_key_iv: "6ec79f6a87f696415176ba492368d425"
server_key_version: "encryptionkey@0"
I20260809 19:29:35.067013 10838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:35.090401 10838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:35.092322 10838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:35.092438 10838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:35.096670 10838 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32823
I20260809 19:29:35.096736 10905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32823 every 8 connection(s)
I20260809 19:29:35.097105 10838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:35.098134 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:35.100454 10906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04: Bootstrap starting.
I20260809 19:29:35.101074 10906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:35.101305 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 10838
I20260809 19:29:35.101418 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:35.101447 10906 log.cc:824] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:35.101668 21831 external_mini_cluster.cc:1428] Setting key 3739e023b927a70c3214a8f34576ad4a
I20260809 19:29:35.102450 10906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04: No bootstrap required, opened a new log
I20260809 19:29:35.105125 10906 raft_consensus.cc:348] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d246e471456a498fa55b7034da709e04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32823 } }
I20260809 19:29:35.105297 10906 raft_consensus.cc:374] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:35.105332 10906 raft_consensus.cc:729] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d246e471456a498fa55b7034da709e04, State: Initialized, Role: FOLLOWER
I20260809 19:29:35.105451 10906 consensus_queue.cc:260] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d246e471456a498fa55b7034da709e04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32823 } }
I20260809 19:29:35.105520 10906 raft_consensus.cc:388] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:35.105547 10906 raft_consensus.cc:482] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:35.105631 10906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:35.106248 10906 raft_consensus.cc:504] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d246e471456a498fa55b7034da709e04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32823 } }
I20260809 19:29:35.106362 10906 leader_election.cc:302] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d246e471456a498fa55b7034da709e04; no voters:
I20260809 19:29:35.106542 10906 leader_election.cc:288] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:35.106640 10911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:35.106819 10911 raft_consensus.cc:686] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [term 1 LEADER]: Becoming Leader. State: Replica: d246e471456a498fa55b7034da709e04, State: Running, Role: LEADER
I20260809 19:29:35.106938 10911 consensus_queue.cc:237] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d246e471456a498fa55b7034da709e04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32823 } }
I20260809 19:29:35.107196 10906 sys_catalog.cc:564] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 19:29:35 dist-test-slave-njxd krb5kdc[10824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 19:29:35.108640 10912 sys_catalog.cc:455] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d246e471456a498fa55b7034da709e04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d246e471456a498fa55b7034da709e04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32823 } } }
I20260809 19:29:35.109284 10912 sys_catalog.cc:458] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:35.108637 10913 sys_catalog.cc:455] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d246e471456a498fa55b7034da709e04. Latest consensus state: current_term: 1 leader_uuid: "d246e471456a498fa55b7034da709e04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d246e471456a498fa55b7034da709e04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32823 } } }
I20260809 19:29:35.109416 10913 sys_catalog.cc:458] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04 [sys.catalog]: This master's current role is: LEADER
W20260809 19:29:35.109774 10929 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:29:35.109833 10929 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:29:35.109938 10931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:35.110140 10931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:35.111647 10931 catalog_manager.cc:1347] Generated new cluster ID: 2d1cf68d6891477a8610874c908d6657
I20260809 19:29:35.111698 10931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:35.113816 10910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:35.102292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38039 (local address 127.0.0.1:32823)
0809 19:29:35.102464 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:35.102474 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:35.102510 (+ 36us) server_negotiation.cc:407] Connection header received
0809 19:29:35.102970 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:35.102977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:35.103024 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:35.103135 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:35.103931 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:35.104771 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:35.105882 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:35.106278 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:35.106461 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:35.109619 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:35.109623 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:35.109624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:35.109635 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:35.112026 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:35.112482 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:35.112485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:35.112486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:35.112596 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:35.112840 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:35.112841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:35.112841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:35.113169 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:35.113351 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:35.113434 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 19:29:35.113558 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260809 19:29:35.114845 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:35.124089 10910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:35.116330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35679 (local address 127.0.0.1:32823)
0809 19:29:35.116365 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:35.116367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:35.116410 (+ 43us) server_negotiation.cc:407] Connection header received
0809 19:29:35.116480 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:35.116481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:35.116491 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:35.116545 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:35.117147 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:35.118004 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:35.119066 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:35.119392 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:35.119434 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:35.120914 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:35.120916 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:35.120916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:35.120923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:35.122555 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:35.122996 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:35.122997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:35.122997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:35.123050 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:35.123294 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:35.123295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:35.123295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:35.123417 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:35.123493 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:35.123909 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 19:29:35.123982 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:29:35.124250 10931 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:35.125012 10931 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:35.140551 10931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d246e471456a498fa55b7034da709e04: Generated new TSK 0
I20260809 19:29:35.140779 10931 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:35.145725 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 10838
W20260809 19:29:35.153023 10821 connection.cc:535] client connection to 127.0.0.1:32823 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32823): Connection reset by peer (error 104)
2026-08-09T19:29:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1359 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:35 dist-test-slave-njxd krb5kdc[10941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:35 dist-test-slave-njxd krb5kdc[10941](info): set up 2 sockets
Aug 09 19:29:35 dist-test-slave-njxd krb5kdc[10941](info): commencing operation
krb5kdc: starting...
W20260809 19:29:36.210060 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:36 dist-test-slave-njxd krb5kdc[10941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:29:36.341882 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39159
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:36.422639 10955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:36.422884 10955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:36.422925 10955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:36.424100 10955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:36.424180 10955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:36.424203 10955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:36.424233 10955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:36.424263 10955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:36.425818 10955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39159
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:36.426103 10955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:36.426347 10955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:36.431216 10964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:36.431216 10961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:36.431370 10955 server_base.cc:1034] running on GCE node
W20260809 19:29:36.431226 10962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:36.431771 10955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:36.432065 10955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:36.433257 10955 hybrid_clock.cc:648] HybridClock initialized: now 1786303776433245 us; error 42 us; skew 500 ppm
Aug 09 19:29:36 dist-test-slave-njxd krb5kdc[10941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303776, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:36.436096 10955 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:29:36.436580 10955 webserver.cc:460] Webserver started at http://127.0.0.1:40607/ using document root <none> and password file <none>
I20260809 19:29:36.436836 10955 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:36.436908 10955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:36.437062 10955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:36.438207 10955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "dbf569c3236048eea84eef20c049fb75"
format_stamp: "Formatted at 2026-08-09 19:29:36 on dist-test-slave-njxd"
server_key: "8f3c76fef5b02b51636c868d30ea333e"
server_key_iv: "ddd3f34ac004b6f1d2ae0d67f1e920a9"
server_key_version: "encryptionkey@0"
I20260809 19:29:36.438570 10955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "dbf569c3236048eea84eef20c049fb75"
format_stamp: "Formatted at 2026-08-09 19:29:36 on dist-test-slave-njxd"
server_key: "8f3c76fef5b02b51636c868d30ea333e"
server_key_iv: "ddd3f34ac004b6f1d2ae0d67f1e920a9"
server_key_version: "encryptionkey@0"
I20260809 19:29:36.439937 10955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 19:29:36.440721 10972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:36.440896 10955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:29:36.441004 10955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "dbf569c3236048eea84eef20c049fb75"
format_stamp: "Formatted at 2026-08-09 19:29:36 on dist-test-slave-njxd"
server_key: "8f3c76fef5b02b51636c868d30ea333e"
server_key_iv: "ddd3f34ac004b6f1d2ae0d67f1e920a9"
server_key_version: "encryptionkey@0"
I20260809 19:29:36.441078 10955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:36.454551 10955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:36.456609 10955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:36.456779 10955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:36.460996 10955 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39159
I20260809 19:29:36.461050 11025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39159 every 8 connection(s)
I20260809 19:29:36.461426 10955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:36.462404 11026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:36.464707 11026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75: Bootstrap starting.
I20260809 19:29:36.465327 11026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:36.465687 11026 log.cc:824] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:36.466405 11026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75: No bootstrap required, opened a new log
I20260809 19:29:36.466529 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 10955
I20260809 19:29:36.466698 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:36.466886 21831 external_mini_cluster.cc:1428] Setting key a5165cd4df9a017b4946aca71ac01914
I20260809 19:29:36.468447 11026 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf569c3236048eea84eef20c049fb75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39159 } }
I20260809 19:29:36.468617 11026 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:36.468657 11026 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbf569c3236048eea84eef20c049fb75, State: Initialized, Role: FOLLOWER
I20260809 19:29:36.468791 11026 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf569c3236048eea84eef20c049fb75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39159 } }
I20260809 19:29:36.468887 11026 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:36.468954 11026 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:36.469007 11026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:36.469625 11026 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf569c3236048eea84eef20c049fb75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39159 } }
I20260809 19:29:36.469775 11026 leader_election.cc:302] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbf569c3236048eea84eef20c049fb75; no voters:
I20260809 19:29:36.469933 11026 leader_election.cc:288] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:36.469960 11031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:36.470067 11031 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [term 1 LEADER]: Becoming Leader. State: Replica: dbf569c3236048eea84eef20c049fb75, State: Running, Role: LEADER
I20260809 19:29:36.470150 11031 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf569c3236048eea84eef20c049fb75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39159 } }
I20260809 19:29:36.470206 11026 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:36.470455 11031 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbf569c3236048eea84eef20c049fb75. Latest consensus state: current_term: 1 leader_uuid: "dbf569c3236048eea84eef20c049fb75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf569c3236048eea84eef20c049fb75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39159 } } }
I20260809 19:29:36.470535 11031 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:36.470469 11032 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbf569c3236048eea84eef20c049fb75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf569c3236048eea84eef20c049fb75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39159 } } }
I20260809 19:29:36.470726 11032 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:36.470896 11037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:36.471410 11037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:36.473886 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:36.467412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54503 (local address 127.0.0.1:39159)
0809 19:29:36.467600 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:36.467608 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:36.467634 (+ 26us) server_negotiation.cc:407] Connection header received
0809 19:29:36.468430 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:36.468436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:36.468482 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:36.468593 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:36.468894 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:36.470470 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:36.471653 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:36.472246 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:36.472354 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:36.473057 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:36.473060 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:36.473061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:36.473272 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:36.473309 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:36.473324 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:36.473408 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:36.473638 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 19:29:36.473669 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260809 19:29:36.474794 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:36.475039 11037 catalog_manager.cc:1347] Generated new cluster ID: 7b0de759eda44401aa9f2cd2850fc6db
I20260809 19:29:36.475095 11037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:36.480290 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:36.476431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42809 (local address 127.0.0.1:39159)
0809 19:29:36.476482 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:36.476484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:36.476615 (+ 131us) server_negotiation.cc:407] Connection header received
0809 19:29:36.476647 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:36.476648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:36.476661 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:36.476731 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:36.476862 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:36.477618 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:36.478584 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:36.479211 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:36.479287 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:36.480036 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:36.480038 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:36.480038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:36.480042 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:36.480059 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:36.480064 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:36.480139 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:36.480201 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 19:29:36.480214 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 19:29:36.486842 11037 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:36.487365 11037 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:36.494910 11037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbf569c3236048eea84eef20c049fb75: Generated new TSK 0
I20260809 19:29:36.495131 11037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:36.498792 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 10955
W20260809 19:29:36.506398 10939 connection.cc:535] client connection to 127.0.0.1:39159 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39159): Connection reset by peer (error 104)
2026-08-09T19:29:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1353 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:36 dist-test-slave-njxd krb5kdc[11058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:36 dist-test-slave-njxd krb5kdc[11058](info): set up 2 sockets
Aug 09 19:29:36 dist-test-slave-njxd krb5kdc[11058](info): commencing operation
krb5kdc: starting...
W20260809 19:29:37.565160 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:37 dist-test-slave-njxd krb5kdc[11058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:29:37.688648 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:37.770327 11071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:37.770589 11071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:37.770628 11071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:37.771792 11071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:37.771863 11071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:37.771886 11071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:37.771907 11071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:37.771936 11071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:37.773427 11071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:37.773747 11071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:37.773983 11071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:37.778882 11080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:37.778909 11078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:37.778942 11071 server_base.cc:1034] running on GCE node
W20260809 19:29:37.778908 11077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:37.779279 11071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:37.779570 11071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:37.780736 11071 hybrid_clock.cc:648] HybridClock initialized: now 1786303777780729 us; error 34 us; skew 500 ppm
Aug 09 19:29:37 dist-test-slave-njxd krb5kdc[11058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303777, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:37.783214 11071 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:29:37.783669 11071 webserver.cc:460] Webserver started at http://127.0.0.1:37647/ using document root <none> and password file <none>
I20260809 19:29:37.783900 11071 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:37.783972 11071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:37.784130 11071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:37.785147 11071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "86ae0d2b756846c6ac0e0f14cc5c612a"
format_stamp: "Formatted at 2026-08-09 19:29:37 on dist-test-slave-njxd"
server_key: "10dad176f5557ff559f9e21a15d2e389"
server_key_iv: "7fb862410ecc1128009ad425f1c1c82c"
server_key_version: "encryptionkey@0"
I20260809 19:29:37.785499 11071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "86ae0d2b756846c6ac0e0f14cc5c612a"
format_stamp: "Formatted at 2026-08-09 19:29:37 on dist-test-slave-njxd"
server_key: "10dad176f5557ff559f9e21a15d2e389"
server_key_iv: "7fb862410ecc1128009ad425f1c1c82c"
server_key_version: "encryptionkey@0"
I20260809 19:29:37.786849 11071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:29:37.787621 11088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:37.787815 11071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:29:37.787923 11071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "86ae0d2b756846c6ac0e0f14cc5c612a"
format_stamp: "Formatted at 2026-08-09 19:29:37 on dist-test-slave-njxd"
server_key: "10dad176f5557ff559f9e21a15d2e389"
server_key_iv: "7fb862410ecc1128009ad425f1c1c82c"
server_key_version: "encryptionkey@0"
I20260809 19:29:37.787976 11071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:37.791358 11071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:37.793504 11071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:37.793679 11071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:37.797799 11071 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42201
I20260809 19:29:37.797829 11144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42201 every 8 connection(s)
I20260809 19:29:37.798169 11071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:37.799146 11145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:37.801429 11145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a: Bootstrap starting.
I20260809 19:29:37.802042 11145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:37.802325 11145 log.cc:824] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:37.803026 11145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a: No bootstrap required, opened a new log
I20260809 19:29:37.803884 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 11071
I20260809 19:29:37.804008 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:37.804210 21831 external_mini_cluster.cc:1428] Setting key 3af0fb5cdf7f55df73d3c8303ff8c9a3
I20260809 19:29:37.805140 11145 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ae0d2b756846c6ac0e0f14cc5c612a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42201 } }
I20260809 19:29:37.805312 11145 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:37.805351 11145 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ae0d2b756846c6ac0e0f14cc5c612a, State: Initialized, Role: FOLLOWER
I20260809 19:29:37.805486 11145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ae0d2b756846c6ac0e0f14cc5c612a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42201 } }
I20260809 19:29:37.805603 11145 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:37.805652 11145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:37.805702 11145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:37.806530 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:37.804738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58967 (local address 127.0.0.1:42201)
0809 19:29:37.804945 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:37.804956 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:37.804992 (+ 36us) server_negotiation.cc:407] Connection header received
0809 19:29:37.805109 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:37.805116 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:37.805146 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:37.805300 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:37.806039 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:37.806042 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:37.806043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:37.806047 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:37.806098 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:37.806115 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:37.806204 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:37.806259 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 19:29:37.806305 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260809 19:29:37.806903 11145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ae0d2b756846c6ac0e0f14cc5c612a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42201 } }
I20260809 19:29:37.807009 11145 leader_election.cc:302] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86ae0d2b756846c6ac0e0f14cc5c612a; no voters:
I20260809 19:29:37.807153 11145 leader_election.cc:288] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:37.807204 11150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:37.807447 11150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [term 1 LEADER]: Becoming Leader. State: Replica: 86ae0d2b756846c6ac0e0f14cc5c612a, State: Running, Role: LEADER
I20260809 19:29:37.807456 11145 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:37.807507 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:37.807595 11150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ae0d2b756846c6ac0e0f14cc5c612a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42201 } }
I20260809 19:29:37.809077 11152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86ae0d2b756846c6ac0e0f14cc5c612a. Latest consensus state: current_term: 1 leader_uuid: "86ae0d2b756846c6ac0e0f14cc5c612a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ae0d2b756846c6ac0e0f14cc5c612a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42201 } } }
I20260809 19:29:37.809265 11151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86ae0d2b756846c6ac0e0f14cc5c612a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ae0d2b756846c6ac0e0f14cc5c612a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42201 } } }
I20260809 19:29:37.809407 11152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:37.809417 11151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a [sys.catalog]: This master's current role is: LEADER
W20260809 19:29:37.809623 11170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:29:37.809672 11170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:29:37.809875 11173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:37.810063 11173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:37.810807 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:37.809116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58981 (local address 127.0.0.1:42201)
0809 19:29:37.809151 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:37.809153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:37.809719 (+ 566us) server_negotiation.cc:407] Connection header received
0809 19:29:37.809814 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:37.809815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:37.809822 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:37.809871 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:37.810562 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:37.810563 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:37.810564 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:37.810568 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:37.810582 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:37.810587 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:37.810617 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:37.810738 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 19:29:37.810754 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:29:37.811494 11173 catalog_manager.cc:1347] Generated new cluster ID: 77b67b5063fc48e09df48d89bc22c6d5
I20260809 19:29:37.811546 11173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:37.841004 11173 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:37.841214 11173 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:37.844225 11173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86ae0d2b756846c6ac0e0f14cc5c612a: Generated new TSK 0
I20260809 19:29:37.844404 11173 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:37.853088 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 11071
2026-08-09T19:29:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1353 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:37 dist-test-slave-njxd krb5kdc[11177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:37 dist-test-slave-njxd krb5kdc[11177](info): set up 2 sockets
Aug 09 19:29:37 dist-test-slave-njxd krb5kdc[11177](info): commencing operation
krb5kdc: starting...
W20260809 19:29:38.916916 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:38 dist-test-slave-njxd krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:29:39.036226 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44537
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:39.120975 11190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:39.121204 11190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:39.121243 11190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:39.122342 11190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:39.122411 11190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:39.122434 11190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:39.122471 11190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:39.122501 11190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:39.123991 11190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:39.124286 11190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:39.124531 11190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:39.129325 11198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:39.129374 11195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:39.129326 11196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:39.129395 11190 server_base.cc:1034] running on GCE node
I20260809 19:29:39.129890 11190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:39.130182 11190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:39.131352 11190 hybrid_clock.cc:648] HybridClock initialized: now 1786303779131345 us; error 37 us; skew 500 ppm
Aug 09 19:29:39 dist-test-slave-njxd krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303779, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:39.133857 11190 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:29:39.134290 11190 webserver.cc:460] Webserver started at http://127.0.0.1:46387/ using document root <none> and password file <none>
I20260809 19:29:39.134536 11190 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:39.134616 11190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:39.134774 11190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:39.135839 11190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "70a4b49c26304cd683ae2128937c4668"
format_stamp: "Formatted at 2026-08-09 19:29:39 on dist-test-slave-njxd"
server_key: "9fb05075e54b4434db809466ecb08d54"
server_key_iv: "b75c85051a62c8d1a2410a6c7e92f285"
server_key_version: "encryptionkey@0"
I20260809 19:29:39.136198 11190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "70a4b49c26304cd683ae2128937c4668"
format_stamp: "Formatted at 2026-08-09 19:29:39 on dist-test-slave-njxd"
server_key: "9fb05075e54b4434db809466ecb08d54"
server_key_iv: "b75c85051a62c8d1a2410a6c7e92f285"
server_key_version: "encryptionkey@0"
I20260809 19:29:39.137543 11190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:29:39.138361 11205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:39.138532 11190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:39.138634 11190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "70a4b49c26304cd683ae2128937c4668"
format_stamp: "Formatted at 2026-08-09 19:29:39 on dist-test-slave-njxd"
server_key: "9fb05075e54b4434db809466ecb08d54"
server_key_iv: "b75c85051a62c8d1a2410a6c7e92f285"
server_key_version: "encryptionkey@0"
I20260809 19:29:39.138716 11190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:39.149315 11190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:39.151314 11190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:39.151484 11190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:39.155733 11190 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44537
I20260809 19:29:39.155783 11262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44537 every 8 connection(s)
I20260809 19:29:39.156126 11190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:39.157006 11263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:39.159308 11263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668: Bootstrap starting.
I20260809 19:29:39.159919 11263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:39.160220 11263 log.cc:824] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:39.160907 11263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668: No bootstrap required, opened a new log
I20260809 19:29:39.161147 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 11190
I20260809 19:29:39.161285 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:39.161466 21831 external_mini_cluster.cc:1428] Setting key b59a7a5fcf616e1ef1aabe4cc69aa77e
I20260809 19:29:39.163260 11263 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a4b49c26304cd683ae2128937c4668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44537 } }
I20260809 19:29:39.163486 11263 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:39.163522 11263 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a4b49c26304cd683ae2128937c4668, State: Initialized, Role: FOLLOWER
I20260809 19:29:39.163651 11263 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a4b49c26304cd683ae2128937c4668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44537 } }
I20260809 19:29:39.163741 11263 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:39.163776 11263 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:39.163826 11263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:39.164438 11263 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a4b49c26304cd683ae2128937c4668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44537 } }
I20260809 19:29:39.164543 11263 leader_election.cc:302] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70a4b49c26304cd683ae2128937c4668; no voters:
I20260809 19:29:39.164700 11263 leader_election.cc:288] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:39.164779 11269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:39.164955 11269 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [term 1 LEADER]: Becoming Leader. State: Replica: 70a4b49c26304cd683ae2128937c4668, State: Running, Role: LEADER
I20260809 19:29:39.164958 11263 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:39.165058 11269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a4b49c26304cd683ae2128937c4668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44537 } }
I20260809 19:29:39.165431 11269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70a4b49c26304cd683ae2128937c4668. Latest consensus state: current_term: 1 leader_uuid: "70a4b49c26304cd683ae2128937c4668" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a4b49c26304cd683ae2128937c4668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44537 } } }
I20260809 19:29:39.165509 11269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:39.165629 11270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70a4b49c26304cd683ae2128937c4668" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a4b49c26304cd683ae2128937c4668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44537 } } }
I20260809 19:29:39.165692 11270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668 [sys.catalog]: This master's current role is: LEADER
W20260809 19:29:39.166978 11284 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:29:39.167953 11284 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:29:39.168038 11285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:39.168226 11285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 19:29:39 dist-test-slave-njxd krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 19:29:39.169709 11285 catalog_manager.cc:1347] Generated new cluster ID: 21be11b54045409b8f76da6261bc99f6
I20260809 19:29:39.169760 11285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:39.173722 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:39.161989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58235 (local address 127.0.0.1:44537)
0809 19:29:39.162213 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:39.162222 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:39.162260 (+ 38us) server_negotiation.cc:407] Connection header received
0809 19:29:39.163491 (+ 1231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:39.163501 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:39.163566 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:39.163719 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:39.164460 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:39.165906 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:39.166749 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:39.167188 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:39.167310 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:39.170121 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:39.170125 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:39.170125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:39.170136 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:39.172075 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:39.172481 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:39.172483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:39.172484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:39.172580 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:39.172822 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:39.172823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:39.172823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:39.173096 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:39.173246 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:39.173364 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 19:29:39.173492 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260809 19:29:39.174734 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:39.179977 11285 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:39.180662 11285 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:39.183172 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:39.176165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34741 (local address 127.0.0.1:44537)
0809 19:29:39.176198 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:39.176199 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:39.176329 (+ 130us) server_negotiation.cc:407] Connection header received
0809 19:29:39.176359 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:39.176360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:39.176370 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:39.176435 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:39.177026 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:39.177783 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:39.178783 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:39.179094 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:39.179167 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:39.180538 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:39.180539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:39.180540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:39.180546 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:39.182120 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:39.182466 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:39.182468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:39.182468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:39.182524 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:39.182706 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:39.182707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:39.182708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:39.182824 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:39.182911 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:39.183005 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 19:29:39.183088 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:29:39.185225 11285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70a4b49c26304cd683ae2128937c4668: Generated new TSK 0
I20260809 19:29:39.185398 11285 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:39.190853 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 11190
2026-08-09T19:29:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:39 dist-test-slave-njxd krb5kdc[11296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:39 dist-test-slave-njxd krb5kdc[11296](info): set up 2 sockets
Aug 09 19:29:39 dist-test-slave-njxd krb5kdc[11296](info): commencing operation
krb5kdc: starting...
W20260809 19:29:40.261364 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:40 dist-test-slave-njxd krb5kdc[11296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:29:40.380890 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45661
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:40.463899 11309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:40.464114 11309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:40.464154 11309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:40.465273 11309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:40.465314 11309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:40.465328 11309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:40.465341 11309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:40.465353 11309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:40.466853 11309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:40.467068 11309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:40.467257 11309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:40.472364 11316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:40.472357 11315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:40.472357 11318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:40.472375 11309 server_base.cc:1034] running on GCE node
I20260809 19:29:40.472882 11309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:40.473210 11309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:40.474383 11309 hybrid_clock.cc:648] HybridClock initialized: now 1786303780474375 us; error 37 us; skew 500 ppm
Aug 09 19:29:40 dist-test-slave-njxd krb5kdc[11296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:40.480068 11309 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:29:40.480582 11309 webserver.cc:460] Webserver started at http://127.0.0.1:35335/ using document root <none> and password file <none>
I20260809 19:29:40.480870 11309 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:40.480943 11309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:40.481101 11309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:40.482268 11309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "a918096c08bc4c0db50a550aa358e9d5"
format_stamp: "Formatted at 2026-08-09 19:29:40 on dist-test-slave-njxd"
server_key: "e934f19adb24ec25142c1d9caf9f7c20"
server_key_iv: "265a459d6ddda9b4db626b7507cc7840"
server_key_version: "encryptionkey@0"
I20260809 19:29:40.482652 11309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "a918096c08bc4c0db50a550aa358e9d5"
format_stamp: "Formatted at 2026-08-09 19:29:40 on dist-test-slave-njxd"
server_key: "e934f19adb24ec25142c1d9caf9f7c20"
server_key_iv: "265a459d6ddda9b4db626b7507cc7840"
server_key_version: "encryptionkey@0"
I20260809 19:29:40.484093 11309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:29:40.484839 11326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:40.485023 11309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:40.485136 11309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "a918096c08bc4c0db50a550aa358e9d5"
format_stamp: "Formatted at 2026-08-09 19:29:40 on dist-test-slave-njxd"
server_key: "e934f19adb24ec25142c1d9caf9f7c20"
server_key_iv: "265a459d6ddda9b4db626b7507cc7840"
server_key_version: "encryptionkey@0"
I20260809 19:29:40.485214 11309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:40.509718 11309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:40.511725 11309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:40.511899 11309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:40.516306 11309 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45661
I20260809 19:29:40.516368 11381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45661 every 8 connection(s)
I20260809 19:29:40.516798 11309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:40.517765 11382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:40.520130 11382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5: Bootstrap starting.
I20260809 19:29:40.520771 11382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:40.521343 11382 log.cc:824] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:40.522178 11382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5: No bootstrap required, opened a new log
I20260809 19:29:40.523905 11382 raft_consensus.cc:348] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a918096c08bc4c0db50a550aa358e9d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45661 } }
I20260809 19:29:40.524063 11382 raft_consensus.cc:374] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:40.524091 11382 raft_consensus.cc:729] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a918096c08bc4c0db50a550aa358e9d5, State: Initialized, Role: FOLLOWER
I20260809 19:29:40.524268 11382 consensus_queue.cc:260] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a918096c08bc4c0db50a550aa358e9d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45661 } }
I20260809 19:29:40.524358 11382 raft_consensus.cc:388] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:40.524384 11382 raft_consensus.cc:482] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:40.524438 11382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:40.525084 11382 raft_consensus.cc:504] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a918096c08bc4c0db50a550aa358e9d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45661 } }
I20260809 19:29:40.525211 11382 leader_election.cc:302] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a918096c08bc4c0db50a550aa358e9d5; no voters:
I20260809 19:29:40.525430 11382 leader_election.cc:288] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:40.525553 11385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:40.525749 11382 sys_catalog.cc:564] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:40.525796 11385 raft_consensus.cc:686] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [term 1 LEADER]: Becoming Leader. State: Replica: a918096c08bc4c0db50a550aa358e9d5, State: Running, Role: LEADER
I20260809 19:29:40.525884 11385 consensus_queue.cc:237] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a918096c08bc4c0db50a550aa358e9d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45661 } }
I20260809 19:29:40.526476 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 11309
I20260809 19:29:40.526600 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:40.526829 21831 external_mini_cluster.cc:1428] Setting key c31edbb0f10ec60f3e0637b685b5560a
I20260809 19:29:40.526816 11386 sys_catalog.cc:455] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a918096c08bc4c0db50a550aa358e9d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a918096c08bc4c0db50a550aa358e9d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45661 } } }
I20260809 19:29:40.526919 11386 sys_catalog.cc:458] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:40.527099 11387 sys_catalog.cc:455] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a918096c08bc4c0db50a550aa358e9d5. Latest consensus state: current_term: 1 leader_uuid: "a918096c08bc4c0db50a550aa358e9d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a918096c08bc4c0db50a550aa358e9d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45661 } } }
I20260809 19:29:40.527191 11387 sys_catalog.cc:458] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5 [sys.catalog]: This master's current role is: LEADER
W20260809 19:29:40.527863 11401 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:29:40.527920 11401 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:29:40.527968 11398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:40.528127 11398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:40.529896 11398 catalog_manager.cc:1347] Generated new cluster ID: a8e85c168150417cbf5536dafdef12c8
I20260809 19:29:40.529958 11398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:40.533349 11405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:40.527601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52481 (local address 127.0.0.1:45661)
0809 19:29:40.527820 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:40.527831 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:40.527866 (+ 35us) server_negotiation.cc:407] Connection header received
0809 19:29:40.527993 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:40.528001 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:40.528061 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:40.528213 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:40.528463 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:40.529770 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:40.530911 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:40.531535 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:40.531625 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:40.532486 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:40.532490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:40.532491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:40.532713 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:40.532750 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:40.532765 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:40.532818 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:40.533020 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 19:29:40.533122 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":78,"threads_started":1}
I20260809 19:29:40.534296 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:40.537261 11398 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:40.537988 11398 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:40.539551 11405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:40.535824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34385 (local address 127.0.0.1:45661)
0809 19:29:40.535882 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:40.535884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:40.536092 (+ 208us) server_negotiation.cc:407] Connection header received
0809 19:29:40.536141 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:40.536142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:40.536397 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:40.536461 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:40.536659 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:40.537147 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:40.538196 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:40.538493 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:40.538527 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:40.539295 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:40.539296 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:40.539297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:40.539301 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:40.539314 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:40.539318 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:40.539343 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:40.539437 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 19:29:40.539479 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 19:29:40.544413 11398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a918096c08bc4c0db50a550aa358e9d5: Generated new TSK 0
I20260809 19:29:40.544613 11398 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:40.547644 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 11309
2026-08-09T19:29:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1356 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:40 dist-test-slave-njxd krb5kdc[11419](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:40 dist-test-slave-njxd krb5kdc[11419](info): set up 2 sockets
Aug 09 19:29:40 dist-test-slave-njxd krb5kdc[11419](info): commencing operation
krb5kdc: starting...
W20260809 19:29:41.616926 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:41 dist-test-slave-njxd krb5kdc[11419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:29:41.736775 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33043
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:41.817800 11432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:41.818013 11432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:41.818038 11432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:41.819319 11432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:41.819362 11432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:41.819392 11432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:41.819404 11432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:41.819458 11432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:41.820938 11432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:41.821189 11432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:41.821417 11432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:41.826272 11438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:41.826300 11441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:41.826328 11439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:41.826424 11432 server_base.cc:1034] running on GCE node
I20260809 19:29:41.826960 11432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:41.827260 11432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:41.828464 11432 hybrid_clock.cc:648] HybridClock initialized: now 1786303781828447 us; error 27 us; skew 500 ppm
Aug 09 19:29:41 dist-test-slave-njxd krb5kdc[11419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:41.831089 11432 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:29:41.831584 11432 webserver.cc:460] Webserver started at http://127.0.0.1:42419/ using document root <none> and password file <none>
I20260809 19:29:41.831815 11432 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:41.831883 11432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:41.832046 11432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:41.833115 11432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "a52d11761ccb4067a927d3bfd1fb90ef"
format_stamp: "Formatted at 2026-08-09 19:29:41 on dist-test-slave-njxd"
server_key: "d884841ef15b9d54e67028cca53b79ef"
server_key_iv: "5d13444f5d61b61e1184a119933145a8"
server_key_version: "encryptionkey@0"
I20260809 19:29:41.833490 11432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "a52d11761ccb4067a927d3bfd1fb90ef"
format_stamp: "Formatted at 2026-08-09 19:29:41 on dist-test-slave-njxd"
server_key: "d884841ef15b9d54e67028cca53b79ef"
server_key_iv: "5d13444f5d61b61e1184a119933145a8"
server_key_version: "encryptionkey@0"
I20260809 19:29:41.834841 11432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:29:41.835572 11448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:41.835745 11432 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:29:41.835850 11432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "a52d11761ccb4067a927d3bfd1fb90ef"
format_stamp: "Formatted at 2026-08-09 19:29:41 on dist-test-slave-njxd"
server_key: "d884841ef15b9d54e67028cca53b79ef"
server_key_iv: "5d13444f5d61b61e1184a119933145a8"
server_key_version: "encryptionkey@0"
I20260809 19:29:41.835935 11432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:41.839217 11432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:41.841485 11432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:41.841672 11432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:41.845798 11432 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33043
I20260809 19:29:41.845826 11502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33043 every 8 connection(s)
I20260809 19:29:41.846153 11432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:41.847092 11503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:41.849390 11503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef: Bootstrap starting.
I20260809 19:29:41.850023 11503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:41.850327 11503 log.cc:824] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:41.851032 11503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef: No bootstrap required, opened a new log
I20260809 19:29:41.851428 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 11432
I20260809 19:29:41.851560 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:41.851754 21831 external_mini_cluster.cc:1428] Setting key f2aeae34db71b77ecc5a02e68f1153c5
I20260809 19:29:41.853461 11503 raft_consensus.cc:348] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52d11761ccb4067a927d3bfd1fb90ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } }
I20260809 19:29:41.853649 11503 raft_consensus.cc:374] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:41.853693 11503 raft_consensus.cc:729] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a52d11761ccb4067a927d3bfd1fb90ef, State: Initialized, Role: FOLLOWER
I20260809 19:29:41.853832 11503 consensus_queue.cc:260] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52d11761ccb4067a927d3bfd1fb90ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } }
I20260809 19:29:41.853943 11503 raft_consensus.cc:388] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:41.853999 11503 raft_consensus.cc:482] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:41.854101 11503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:41.854311 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:41.852237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53475 (local address 127.0.0.1:33043)
0809 19:29:41.852419 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:41.852426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:41.852451 (+ 25us) server_negotiation.cc:407] Connection header received
0809 19:29:41.852986 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:41.852992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:41.853014 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:41.853120 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:41.853835 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:41.853838 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:41.853839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:41.853843 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:41.853891 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:41.853900 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:41.853978 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:41.854077 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 19:29:41.854119 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":100,"threads_started":1}
I20260809 19:29:41.855170 11503 raft_consensus.cc:504] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52d11761ccb4067a927d3bfd1fb90ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } }
I20260809 19:29:41.855257 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:41.855304 11503 leader_election.cc:302] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a52d11761ccb4067a927d3bfd1fb90ef; no voters:
I20260809 19:29:41.855451 11503 leader_election.cc:288] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:41.855523 11508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:41.855669 11503 sys_catalog.cc:564] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:41.855829 11508 raft_consensus.cc:686] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [term 1 LEADER]: Becoming Leader. State: Replica: a52d11761ccb4067a927d3bfd1fb90ef, State: Running, Role: LEADER
I20260809 19:29:41.855924 11508 consensus_queue.cc:237] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52d11761ccb4067a927d3bfd1fb90ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } }
I20260809 19:29:41.856295 11508 sys_catalog.cc:455] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader a52d11761ccb4067a927d3bfd1fb90ef. Latest consensus state: current_term: 1 leader_uuid: "a52d11761ccb4067a927d3bfd1fb90ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52d11761ccb4067a927d3bfd1fb90ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } } }
I20260809 19:29:41.856307 11509 sys_catalog.cc:455] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a52d11761ccb4067a927d3bfd1fb90ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52d11761ccb4067a927d3bfd1fb90ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } } }
I20260809 19:29:41.856451 11509 sys_catalog.cc:458] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:41.856663 11508 sys_catalog.cc:458] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:41.856806 11525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:41.857322 11525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:41.859092 11525 catalog_manager.cc:1347] Generated new cluster ID: 5d1e176b5d434fd8abcaf87d9a75f605
I20260809 19:29:41.859166 11525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:41.859182 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:41.857568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40107 (local address 127.0.0.1:33043)
0809 19:29:41.857659 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:41.857661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:41.857933 (+ 272us) server_negotiation.cc:407] Connection header received
0809 19:29:41.858045 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:41.858046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:41.858051 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:41.858118 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:41.858854 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:41.858855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:41.858855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:41.858859 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:41.858873 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:41.858876 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:41.858916 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:41.859029 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 19:29:41.859042 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 19:29:41.865404 11525 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:41.865602 11525 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:41.869405 11525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a52d11761ccb4067a927d3bfd1fb90ef: Generated new TSK 0
I20260809 19:29:41.869618 11525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:41.871518 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 11432
2026-08-09T19:29:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1323 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:41 dist-test-slave-njxd krb5kdc[11538](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:41 dist-test-slave-njxd krb5kdc[11538](info): set up 2 sockets
Aug 09 19:29:41 dist-test-slave-njxd krb5kdc[11538](info): commencing operation
krb5kdc: starting...
W20260809 19:29:42.932616 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:43 dist-test-slave-njxd krb5kdc[11538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:29:43.054241 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46021
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:43.138891 11553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:43.139094 11553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:43.139117 11553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:43.140151 11553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:43.140182 11553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:43.140195 11553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:43.140208 11553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:43.140218 11553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:43.141633 11553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:43.141867 11553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:43.142163 11553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:43.147006 11559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:43.147006 11562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:43.147025 11553 server_base.cc:1034] running on GCE node
W20260809 19:29:43.147042 11558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:43.147483 11553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:43.147779 11553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:43.148952 11553 hybrid_clock.cc:648] HybridClock initialized: now 1786303783148936 us; error 43 us; skew 500 ppm
Aug 09 19:29:43 dist-test-slave-njxd krb5kdc[11538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303783, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:43.151496 11553 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:29:43.151943 11553 webserver.cc:460] Webserver started at http://127.0.0.1:45287/ using document root <none> and password file <none>
I20260809 19:29:43.152122 11553 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:43.152168 11553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:43.152258 11553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:43.153246 11553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "852afc6cf2054ef09e8de4cbc02e6f45"
format_stamp: "Formatted at 2026-08-09 19:29:43 on dist-test-slave-njxd"
server_key: "cd85695b22ffdcfb542a3b4ad79378a7"
server_key_iv: "a10e408ea766f4ce49e89f7a3712d464"
server_key_version: "encryptionkey@0"
I20260809 19:29:43.153542 11553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "852afc6cf2054ef09e8de4cbc02e6f45"
format_stamp: "Formatted at 2026-08-09 19:29:43 on dist-test-slave-njxd"
server_key: "cd85695b22ffdcfb542a3b4ad79378a7"
server_key_iv: "a10e408ea766f4ce49e89f7a3712d464"
server_key_version: "encryptionkey@0"
I20260809 19:29:43.154917 11553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:29:43.155587 11569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:43.155776 11553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:29:43.155864 11553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "852afc6cf2054ef09e8de4cbc02e6f45"
format_stamp: "Formatted at 2026-08-09 19:29:43 on dist-test-slave-njxd"
server_key: "cd85695b22ffdcfb542a3b4ad79378a7"
server_key_iv: "a10e408ea766f4ce49e89f7a3712d464"
server_key_version: "encryptionkey@0"
I20260809 19:29:43.155920 11553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:43.182178 11553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:43.184340 11553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:43.184458 11553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:43.188453 11553 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46021
I20260809 19:29:43.188557 11623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46021 every 8 connection(s)
I20260809 19:29:43.188865 11553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:43.189316 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 11553
I20260809 19:29:43.189445 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:43.189667 21831 external_mini_cluster.cc:1428] Setting key e7af437108d5f6d17e001160fdb9528d
I20260809 19:29:43.190025 11624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:43.193451 11624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45: Bootstrap starting.
I20260809 19:29:43.194180 11624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:43.194490 11624 log.cc:824] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:43.195379 11624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45: No bootstrap required, opened a new log
Aug 09 19:29:43 dist-test-slave-njxd krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 19:29:43.197351 11624 raft_consensus.cc:348] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "852afc6cf2054ef09e8de4cbc02e6f45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46021 } }
I20260809 19:29:43.197536 11624 raft_consensus.cc:374] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:43.197614 11624 raft_consensus.cc:729] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 852afc6cf2054ef09e8de4cbc02e6f45, State: Initialized, Role: FOLLOWER
I20260809 19:29:43.197759 11624 consensus_queue.cc:260] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "852afc6cf2054ef09e8de4cbc02e6f45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46021 } }
I20260809 19:29:43.197857 11624 raft_consensus.cc:388] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:43.197901 11624 raft_consensus.cc:482] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:43.197947 11624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:43.198423 11624 raft_consensus.cc:504] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "852afc6cf2054ef09e8de4cbc02e6f45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46021 } }
I20260809 19:29:43.198544 11624 leader_election.cc:302] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 852afc6cf2054ef09e8de4cbc02e6f45; no voters:
I20260809 19:29:43.198750 11624 leader_election.cc:288] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:43.198791 11629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:43.198900 11629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [term 1 LEADER]: Becoming Leader. State: Replica: 852afc6cf2054ef09e8de4cbc02e6f45, State: Running, Role: LEADER
I20260809 19:29:43.198979 11629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "852afc6cf2054ef09e8de4cbc02e6f45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46021 } }
I20260809 19:29:43.199004 11624 sys_catalog.cc:564] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:43.199497 11629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "852afc6cf2054ef09e8de4cbc02e6f45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "852afc6cf2054ef09e8de4cbc02e6f45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46021 } } }
I20260809 19:29:43.199611 11629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:43.199787 11629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 852afc6cf2054ef09e8de4cbc02e6f45. Latest consensus state: current_term: 1 leader_uuid: "852afc6cf2054ef09e8de4cbc02e6f45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "852afc6cf2054ef09e8de4cbc02e6f45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46021 } } }
I20260809 19:29:43.199914 11629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:43.200384 11637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:43.200889 11637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:43.201443 11627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:43.190207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:43063 (local address 127.0.0.1:46021)
0809 19:29:43.190374 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:43.190383 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:43.190416 (+ 33us) server_negotiation.cc:407] Connection header received
0809 19:29:43.190978 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:43.190987 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:43.191053 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:43.191210 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:43.191792 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:43.193175 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:43.194038 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:43.194711 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:43.194815 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:43.197124 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:43.197128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:43.197128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:43.197145 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:43.199274 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:43.199832 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:43.199835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:43.199835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:43.199935 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:43.200237 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:43.200239 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:43.200239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:43.200544 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:43.200713 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:43.201134 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 19:29:43.201242 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260809 19:29:43.202328 11637 catalog_manager.cc:1347] Generated new cluster ID: cb6b6a193fa3451a84d1396598e02e2e
I20260809 19:29:43.202379 11637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:43.202389 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:43.209857 11627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:43.203741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:60579 (local address 127.0.0.1:46021)
0809 19:29:43.203777 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:43.203780 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:43.203941 (+ 161us) server_negotiation.cc:407] Connection header received
0809 19:29:43.204000 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:43.204001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:43.204010 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:43.204066 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:43.204561 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:43.205014 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:43.205760 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:43.206005 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:43.206079 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:43.207253 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:43.207254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:43.207254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:43.207262 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:43.208661 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:43.209030 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:43.209031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:43.209031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:43.209102 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:43.209339 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:43.209340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:43.209341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:43.209451 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:43.209557 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:43.209704 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 19:29:43.209767 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:29:43.214778 11637 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:43.215278 11637 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:43.222595 11637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 852afc6cf2054ef09e8de4cbc02e6f45: Generated new TSK 0
I20260809 19:29:43.222810 11637 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:43.231542 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 11553
W20260809 19:29:43.238101 11536 connection.cc:535] client connection to 127.0.0.1:46021 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46021): Connection reset by peer (error 104)
2026-08-09T19:29:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1359 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:43 dist-test-slave-njxd krb5kdc[11658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:43 dist-test-slave-njxd krb5kdc[11658](info): set up 2 sockets
Aug 09 19:29:43 dist-test-slave-njxd krb5kdc[11658](info): commencing operation
krb5kdc: starting...
W20260809 19:29:44.281783 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:44 dist-test-slave-njxd krb5kdc[11658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:29:44.400256 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34011
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:44.481822 11672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:44.482075 11672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:44.482116 11672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:44.483268 11672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:44.483345 11672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:44.483368 11672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:44.483389 11672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:44.483419 11672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:44.484952 11672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:44.485241 11672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:44.485482 11672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:44.490417 11679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:44.490528 11672 server_base.cc:1034] running on GCE node
W20260809 19:29:44.490423 11678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:44.490423 11681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:44.490919 11672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:44.491228 11672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:44.492374 11672 hybrid_clock.cc:648] HybridClock initialized: now 1786303784492363 us; error 32 us; skew 500 ppm
Aug 09 19:29:44 dist-test-slave-njxd krb5kdc[11658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303784, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:44.494843 11672 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:29:44.495290 11672 webserver.cc:460] Webserver started at http://127.0.0.1:37149/ using document root <none> and password file <none>
I20260809 19:29:44.495525 11672 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:44.495594 11672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:44.495747 11672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:44.496776 11672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "6e0ee8fa87dc41eab20137ac7f5556fb"
format_stamp: "Formatted at 2026-08-09 19:29:44 on dist-test-slave-njxd"
server_key: "a83dc056774c44c5ab4a645b8cb4295f"
server_key_iv: "13798cd5c62e1b14aa7526d7fff47cde"
server_key_version: "encryptionkey@0"
I20260809 19:29:44.497134 11672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "6e0ee8fa87dc41eab20137ac7f5556fb"
format_stamp: "Formatted at 2026-08-09 19:29:44 on dist-test-slave-njxd"
server_key: "a83dc056774c44c5ab4a645b8cb4295f"
server_key_iv: "13798cd5c62e1b14aa7526d7fff47cde"
server_key_version: "encryptionkey@0"
I20260809 19:29:44.498531 11672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:29:44.499202 11688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:44.499408 11672 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:44.499511 11672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "6e0ee8fa87dc41eab20137ac7f5556fb"
format_stamp: "Formatted at 2026-08-09 19:29:44 on dist-test-slave-njxd"
server_key: "a83dc056774c44c5ab4a645b8cb4295f"
server_key_iv: "13798cd5c62e1b14aa7526d7fff47cde"
server_key_version: "encryptionkey@0"
I20260809 19:29:44.499581 11672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:44.509099 11672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:44.510996 11672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:44.511157 11672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:44.515178 11672 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34011
I20260809 19:29:44.515210 11744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34011 every 8 connection(s)
I20260809 19:29:44.515539 11672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:44.516426 11745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:44.518720 11745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb: Bootstrap starting.
I20260809 19:29:44.519292 11745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:44.519587 11745 log.cc:824] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:44.520257 11745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb: No bootstrap required, opened a new log
I20260809 19:29:44.521930 11745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0ee8fa87dc41eab20137ac7f5556fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34011 } }
I20260809 19:29:44.522069 11745 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:44.522115 11745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0ee8fa87dc41eab20137ac7f5556fb, State: Initialized, Role: FOLLOWER
I20260809 19:29:44.522279 11745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0ee8fa87dc41eab20137ac7f5556fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34011 } }
I20260809 19:29:44.522367 11745 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:44.522420 11745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:44.522473 11745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:44.522990 11745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0ee8fa87dc41eab20137ac7f5556fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34011 } }
I20260809 19:29:44.523101 11745 leader_election.cc:302] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e0ee8fa87dc41eab20137ac7f5556fb; no voters:
I20260809 19:29:44.523281 11745 leader_election.cc:288] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:44.523303 11750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:44.523475 11750 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [term 1 LEADER]: Becoming Leader. State: Replica: 6e0ee8fa87dc41eab20137ac7f5556fb, State: Running, Role: LEADER
I20260809 19:29:44.523537 11745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:44.523615 11750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0ee8fa87dc41eab20137ac7f5556fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34011 } }
I20260809 19:29:44.524003 11752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e0ee8fa87dc41eab20137ac7f5556fb. Latest consensus state: current_term: 1 leader_uuid: "6e0ee8fa87dc41eab20137ac7f5556fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0ee8fa87dc41eab20137ac7f5556fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34011 } } }
I20260809 19:29:44.523998 11751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e0ee8fa87dc41eab20137ac7f5556fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0ee8fa87dc41eab20137ac7f5556fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34011 } } }
I20260809 19:29:44.524107 11752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:44.524170 11751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:44.524848 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 11672
I20260809 19:29:44.524937 11754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:44.524994 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:44.525220 21831 external_mini_cluster.cc:1428] Setting key 8217ea7c5d666eef81604e71a69e0375
I20260809 19:29:44.525490 11754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:44.530097 11754 catalog_manager.cc:1347] Generated new cluster ID: 9ae901bf98614bad9d0ec5ad5a051e43
I20260809 19:29:44.530150 11754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:44.534539 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:44.525840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:44019 (local address 127.0.0.1:34011)
0809 19:29:44.526026 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:44.526034 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:44.526105 (+ 71us) server_negotiation.cc:407] Connection header received
0809 19:29:44.526262 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:44.526269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:44.526325 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:44.526468 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:44.526791 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:44.528133 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:44.528899 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:44.529504 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:44.529639 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:44.533847 (+ 4208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:44.533851 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:44.533852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:44.534063 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:44.534100 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:44.534114 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:44.534204 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:44.534296 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 19:29:44.534324 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260809 19:29:44.535591 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:44.541126 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:44.537084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:42389 (local address 127.0.0.1:34011)
0809 19:29:44.537125 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:44.537126 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:44.537756 (+ 630us) server_negotiation.cc:407] Connection header received
0809 19:29:44.537813 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:44.537813 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:44.537843 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:44.537888 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:44.538028 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:44.538748 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:44.539789 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:44.540088 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:44.540155 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:44.540875 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:44.540876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:44.540876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:44.540880 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:44.540895 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:44.540900 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:44.540955 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:44.541036 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 19:29:44.541050 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:29:44.550890 11754 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:44.551455 11754 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:44.554453 11754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e0ee8fa87dc41eab20137ac7f5556fb: Generated new TSK 0
I20260809 19:29:44.554672 11754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:44.562868 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 11672
W20260809 19:29:44.570391 11656 connection.cc:535] client connection to 127.0.0.1:34011 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34011): Connection reset by peer (error 104)
2026-08-09T19:29:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:44 dist-test-slave-njxd krb5kdc[11779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:44 dist-test-slave-njxd krb5kdc[11779](info): set up 2 sockets
Aug 09 19:29:44 dist-test-slave-njxd krb5kdc[11779](info): commencing operation
krb5kdc: starting...
W20260809 19:29:45.625418 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:45 dist-test-slave-njxd krb5kdc[11779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:29:45.745934 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46181
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:45.827571 11793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:45.827819 11793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:45.827858 11793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:45.829032 11793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:45.829098 11793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:45.829123 11793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:45.829152 11793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:45.829182 11793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:45.830739 11793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:45.831032 11793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:45.831290 11793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:45.836162 11801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:45.836155 11798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:45.836165 11799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:45.836391 11793 server_base.cc:1034] running on GCE node
I20260809 19:29:45.836678 11793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:45.837004 11793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:45.838163 11793 hybrid_clock.cc:648] HybridClock initialized: now 1786303785838157 us; error 34 us; skew 500 ppm
Aug 09 19:29:45 dist-test-slave-njxd krb5kdc[11779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:45.846560 11793 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:29:45.847049 11793 webserver.cc:460] Webserver started at http://127.0.0.1:43435/ using document root <none> and password file <none>
I20260809 19:29:45.847308 11793 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:45.847379 11793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:45.847529 11793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:45.848567 11793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "8a4e829cb61e4448902894cfb1eee146"
format_stamp: "Formatted at 2026-08-09 19:29:45 on dist-test-slave-njxd"
server_key: "790b144139bfbb8d3e65d63943e042a0"
server_key_iv: "fc0b1f4be9e7941eaa8ec0a1448b8dfa"
server_key_version: "encryptionkey@0"
I20260809 19:29:45.848964 11793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "8a4e829cb61e4448902894cfb1eee146"
format_stamp: "Formatted at 2026-08-09 19:29:45 on dist-test-slave-njxd"
server_key: "790b144139bfbb8d3e65d63943e042a0"
server_key_iv: "fc0b1f4be9e7941eaa8ec0a1448b8dfa"
server_key_version: "encryptionkey@0"
I20260809 19:29:45.850407 11793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:29:45.851226 11808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:45.851401 11793 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:45.851492 11793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "8a4e829cb61e4448902894cfb1eee146"
format_stamp: "Formatted at 2026-08-09 19:29:45 on dist-test-slave-njxd"
server_key: "790b144139bfbb8d3e65d63943e042a0"
server_key_iv: "fc0b1f4be9e7941eaa8ec0a1448b8dfa"
server_key_version: "encryptionkey@0"
I20260809 19:29:45.851545 11793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:45.854774 11793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:45.856956 11793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:45.857060 11793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:45.861022 11793 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46181
I20260809 19:29:45.861100 11865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46181 every 8 connection(s)
I20260809 19:29:45.861399 11793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:45.862465 11866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:45.864668 11866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146: Bootstrap starting.
I20260809 19:29:45.865264 11866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:45.865551 11866 log.cc:824] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:45.866282 11866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146: No bootstrap required, opened a new log
I20260809 19:29:45.868028 11866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4e829cb61e4448902894cfb1eee146" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46181 } }
I20260809 19:29:45.868185 11866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:45.868263 11866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a4e829cb61e4448902894cfb1eee146, State: Initialized, Role: FOLLOWER
I20260809 19:29:45.868451 11866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4e829cb61e4448902894cfb1eee146" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46181 } }
I20260809 19:29:45.868548 11866 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:45.868592 11866 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:45.868640 11866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:45.869140 11866 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4e829cb61e4448902894cfb1eee146" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46181 } }
I20260809 19:29:45.869262 11866 leader_election.cc:302] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a4e829cb61e4448902894cfb1eee146; no voters:
I20260809 19:29:45.869432 11866 leader_election.cc:288] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:45.869457 11869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:45.869645 11869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [term 1 LEADER]: Becoming Leader. State: Replica: 8a4e829cb61e4448902894cfb1eee146, State: Running, Role: LEADER
I20260809 19:29:45.869693 11866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:45.869732 11869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4e829cb61e4448902894cfb1eee146" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46181 } }
I20260809 19:29:45.870151 11871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a4e829cb61e4448902894cfb1eee146. Latest consensus state: current_term: 1 leader_uuid: "8a4e829cb61e4448902894cfb1eee146" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4e829cb61e4448902894cfb1eee146" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46181 } } }
I20260809 19:29:45.870231 11871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:45.870282 11870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a4e829cb61e4448902894cfb1eee146" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4e829cb61e4448902894cfb1eee146" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46181 } } }
I20260809 19:29:45.870375 11870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:45.870589 11877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:45.871114 11877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:45.871137 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 11793
I20260809 19:29:45.871261 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:45.871433 21831 external_mini_cluster.cc:1428] Setting key 53213e6b139591a7144ffc1369ca688a
I20260809 19:29:45.872869 11877 catalog_manager.cc:1347] Generated new cluster ID: 549cf17543584ab0a872f608cd31049f
I20260809 19:29:45.872956 11877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:45.873911 11889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:45.872031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:43777 (local address 127.0.0.1:46181)
0809 19:29:45.872258 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:45.872266 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:45.872298 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:29:45.872430 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:45.872437 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:45.872467 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:45.872640 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:45.873357 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:45.873360 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:45.873361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:45.873365 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:45.873417 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:45.873432 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:45.873506 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:45.873593 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 19:29:45.873737 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":77,"threads_started":1}
I20260809 19:29:45.874876 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:45.877601 11889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:45.876524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:40521 (local address 127.0.0.1:46181)
0809 19:29:45.876565 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:45.876567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:45.876715 (+ 148us) server_negotiation.cc:407] Connection header received
0809 19:29:45.876770 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:45.876771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:45.876776 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:45.876821 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:45.877385 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:45.877386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:45.877386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:45.877389 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:45.877401 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:45.877405 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:45.877453 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:45.877507 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 19:29:45.877518 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:29:45.904925 11877 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:45.905153 11877 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:45.908799 11877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a4e829cb61e4448902894cfb1eee146: Generated new TSK 0
I20260809 19:29:45.909009 11877 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:45.911281 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 11793
2026-08-09T19:29:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1348 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:45 dist-test-slave-njxd krb5kdc[11898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:45 dist-test-slave-njxd krb5kdc[11898](info): set up 2 sockets
Aug 09 19:29:45 dist-test-slave-njxd krb5kdc[11898](info): commencing operation
krb5kdc: starting...
W20260809 19:29:46.982355 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:47 dist-test-slave-njxd krb5kdc[11898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:29:47.106648 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40013
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:47.188117 11911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:47.188334 11911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:47.188359 11911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:47.189519 11911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:47.189561 11911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:47.189592 11911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:47.189604 11911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:47.189616 11911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:47.191184 11911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40013
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:47.191430 11911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:47.191664 11911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:47.196583 11917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:47.196564 11920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:47.196563 11918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:47.196717 11911 server_base.cc:1034] running on GCE node
I20260809 19:29:47.197160 11911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:47.197458 11911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:47.198624 11911 hybrid_clock.cc:648] HybridClock initialized: now 1786303787198601 us; error 55 us; skew 500 ppm
Aug 09 19:29:47 dist-test-slave-njxd krb5kdc[11898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303787, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:47.201195 11911 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:29:47.201674 11911 webserver.cc:460] Webserver started at http://127.0.0.1:44967/ using document root <none> and password file <none>
I20260809 19:29:47.201907 11911 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:47.201973 11911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:47.202143 11911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:47.203295 11911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "e5604f4494e34688b027535dec82be35"
format_stamp: "Formatted at 2026-08-09 19:29:47 on dist-test-slave-njxd"
server_key: "9bfa1ae5b70b2bf305f457f15647bffd"
server_key_iv: "9397d82f21c62da4eb9a1e271b72a0b2"
server_key_version: "encryptionkey@0"
I20260809 19:29:47.203665 11911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "e5604f4494e34688b027535dec82be35"
format_stamp: "Formatted at 2026-08-09 19:29:47 on dist-test-slave-njxd"
server_key: "9bfa1ae5b70b2bf305f457f15647bffd"
server_key_iv: "9397d82f21c62da4eb9a1e271b72a0b2"
server_key_version: "encryptionkey@0"
I20260809 19:29:47.205035 11911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:29:47.205837 11927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:47.206008 11911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:29:47.206120 11911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "e5604f4494e34688b027535dec82be35"
format_stamp: "Formatted at 2026-08-09 19:29:47 on dist-test-slave-njxd"
server_key: "9bfa1ae5b70b2bf305f457f15647bffd"
server_key_iv: "9397d82f21c62da4eb9a1e271b72a0b2"
server_key_version: "encryptionkey@0"
I20260809 19:29:47.206195 11911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:47.265030 11911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:47.267529 11911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:47.267711 11911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:47.272023 11911 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40013
I20260809 19:29:47.272080 11981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40013 every 8 connection(s)
I20260809 19:29:47.272486 11911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:47.273433 11982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:47.275813 11982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35: Bootstrap starting.
I20260809 19:29:47.276434 11982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:47.276726 11982 log.cc:824] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:47.277443 11982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35: No bootstrap required, opened a new log
I20260809 19:29:47.279043 11982 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5604f4494e34688b027535dec82be35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40013 } }
I20260809 19:29:47.279191 11982 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:47.279214 11982 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5604f4494e34688b027535dec82be35, State: Initialized, Role: FOLLOWER
I20260809 19:29:47.279392 11982 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5604f4494e34688b027535dec82be35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40013 } }
I20260809 19:29:47.279454 11982 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:47.279475 11982 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:47.279546 11982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:47.280071 11982 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5604f4494e34688b027535dec82be35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40013 } }
I20260809 19:29:47.280192 11982 leader_election.cc:302] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5604f4494e34688b027535dec82be35; no voters:
I20260809 19:29:47.280390 11982 leader_election.cc:288] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:47.280452 11985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:47.280632 11985 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [term 1 LEADER]: Becoming Leader. State: Replica: e5604f4494e34688b027535dec82be35, State: Running, Role: LEADER
I20260809 19:29:47.280650 11982 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:47.280728 11985 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5604f4494e34688b027535dec82be35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40013 } }
I20260809 19:29:47.281322 11986 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5604f4494e34688b027535dec82be35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5604f4494e34688b027535dec82be35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40013 } } }
I20260809 19:29:47.281504 11987 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5604f4494e34688b027535dec82be35. Latest consensus state: current_term: 1 leader_uuid: "e5604f4494e34688b027535dec82be35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5604f4494e34688b027535dec82be35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40013 } } }
I20260809 19:29:47.281643 11987 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:47.281591 11986 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:47.281944 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 11911
I20260809 19:29:47.281971 11996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:47.282070 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:47.282292 21831 external_mini_cluster.cc:1428] Setting key b1d030cf9d2101d92fde7ddb7c6d95d7
I20260809 19:29:47.282591 11996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:47.284675 11996 catalog_manager.cc:1347] Generated new cluster ID: c7c6edf64dfc48aa9b276064c59c2d63
I20260809 19:29:47.284735 11996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:29:47 dist-test-slave-njxd krb5kdc[11898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 19:29:47.293963 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:47.282958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:48765 (local address 127.0.0.1:40013)
0809 19:29:47.283130 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:47.283139 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:47.283170 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:29:47.283284 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:47.283290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:47.283344 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:47.283477 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:47.284328 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:47.285649 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:47.286718 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:47.287314 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:47.287402 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:47.290191 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:47.290195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:47.290195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:47.290207 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:47.292150 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:47.292580 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:47.292583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:47.292583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:47.292725 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:47.292956 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:47.292957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:47.292957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:47.293250 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:47.293417 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:47.293495 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 19:29:47.293745 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
I20260809 19:29:47.295033 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:47.303685 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:47.296503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:49749 (local address 127.0.0.1:40013)
0809 19:29:47.296537 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:47.296539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:47.296662 (+ 123us) server_negotiation.cc:407] Connection header received
0809 19:29:47.296727 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:47.296727 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:47.296737 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:47.296787 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:47.297396 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:47.298179 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:47.299177 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:47.299539 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:47.299595 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:47.300946 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:47.300947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:47.300947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:47.300954 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:47.302480 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:47.302914 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:47.302915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:47.302915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:47.302968 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:47.303196 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:47.303197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:47.303197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:47.303306 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:47.303394 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:47.303544 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 19:29:47.303603 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:29:47.306705 11996 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:47.307161 11996 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:47.311275 11996 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5604f4494e34688b027535dec82be35: Generated new TSK 0
I20260809 19:29:47.311453 11996 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:47.315354 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 11911
W20260809 19:29:47.322157 11896 connection.cc:535] client connection to 127.0.0.1:40013 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40013): Connection reset by peer (error 104)
2026-08-09T19:29:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:47 dist-test-slave-njxd krb5kdc[12016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:47 dist-test-slave-njxd krb5kdc[12016](info): set up 2 sockets
Aug 09 19:29:47 dist-test-slave-njxd krb5kdc[12016](info): commencing operation
krb5kdc: starting...
W20260809 19:29:48.364455 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:48 dist-test-slave-njxd krb5kdc[12016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:29:48.484154 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33569
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:48.566867 12030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:48.567077 12030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:48.567099 12030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:48.568275 12030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:48.568320 12030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:48.568333 12030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:48.568346 12030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:48.568398 12030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:48.569967 12030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33569
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:48.570241 12030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:48.570479 12030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:48.575667 12040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:48.575593 12037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:48.575593 12036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:48.575899 12030 server_base.cc:1034] running on GCE node
I20260809 19:29:48.576143 12030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:48.576485 12030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:48.577641 12030 hybrid_clock.cc:648] HybridClock initialized: now 1786303788577620 us; error 39 us; skew 500 ppm
Aug 09 19:29:48 dist-test-slave-njxd krb5kdc[12016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303788, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:48.580068 12030 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:29:48.580524 12030 webserver.cc:460] Webserver started at http://127.0.0.1:35833/ using document root <none> and password file <none>
I20260809 19:29:48.580753 12030 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:48.580822 12030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:48.580976 12030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:48.582118 12030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "e39385b3dc1a4ffb95959aad665e89f8"
format_stamp: "Formatted at 2026-08-09 19:29:48 on dist-test-slave-njxd"
server_key: "b255dd57753fc64da649c66b90ffc597"
server_key_iv: "4aec8517fa7a016639ed7fe73e633c96"
server_key_version: "encryptionkey@0"
I20260809 19:29:48.582489 12030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "e39385b3dc1a4ffb95959aad665e89f8"
format_stamp: "Formatted at 2026-08-09 19:29:48 on dist-test-slave-njxd"
server_key: "b255dd57753fc64da649c66b90ffc597"
server_key_iv: "4aec8517fa7a016639ed7fe73e633c96"
server_key_version: "encryptionkey@0"
I20260809 19:29:48.583825 12030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:29:48.584570 12048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:48.584771 12030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:48.584893 12030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "e39385b3dc1a4ffb95959aad665e89f8"
format_stamp: "Formatted at 2026-08-09 19:29:48 on dist-test-slave-njxd"
server_key: "b255dd57753fc64da649c66b90ffc597"
server_key_iv: "4aec8517fa7a016639ed7fe73e633c96"
server_key_version: "encryptionkey@0"
I20260809 19:29:48.584964 12030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:48.595705 12030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:48.597491 12030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:48.597676 12030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:48.601536 12030 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33569
I20260809 19:29:48.601609 12102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33569 every 8 connection(s)
I20260809 19:29:48.601981 12030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:48.602885 12103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:48.605155 12103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8: Bootstrap starting.
I20260809 19:29:48.605778 12103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:48.606020 12103 log.cc:824] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:48.606699 12103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8: No bootstrap required, opened a new log
I20260809 19:29:48.608263 12103 raft_consensus.cc:348] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e39385b3dc1a4ffb95959aad665e89f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } }
I20260809 19:29:48.608407 12103 raft_consensus.cc:374] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:48.608429 12103 raft_consensus.cc:729] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e39385b3dc1a4ffb95959aad665e89f8, State: Initialized, Role: FOLLOWER
I20260809 19:29:48.608556 12103 consensus_queue.cc:260] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e39385b3dc1a4ffb95959aad665e89f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } }
I20260809 19:29:48.608636 12103 raft_consensus.cc:388] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:48.608659 12103 raft_consensus.cc:482] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:48.608687 12103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:48.609210 12103 raft_consensus.cc:504] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e39385b3dc1a4ffb95959aad665e89f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } }
I20260809 19:29:48.609262 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 12030
I20260809 19:29:48.609345 12103 leader_election.cc:302] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e39385b3dc1a4ffb95959aad665e89f8; no voters:
I20260809 19:29:48.609395 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:48.609524 12103 leader_election.cc:288] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:48.609575 21831 external_mini_cluster.cc:1428] Setting key 987ff77d5f15ec678c63ec41bad5efbd
I20260809 19:29:48.609558 12106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:48.609752 12106 raft_consensus.cc:686] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [term 1 LEADER]: Becoming Leader. State: Replica: e39385b3dc1a4ffb95959aad665e89f8, State: Running, Role: LEADER
I20260809 19:29:48.609813 12103 sys_catalog.cc:564] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:48.610460 12106 consensus_queue.cc:237] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e39385b3dc1a4ffb95959aad665e89f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } }
I20260809 19:29:48.611158 12108 sys_catalog.cc:455] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e39385b3dc1a4ffb95959aad665e89f8. Latest consensus state: current_term: 1 leader_uuid: "e39385b3dc1a4ffb95959aad665e89f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e39385b3dc1a4ffb95959aad665e89f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } } }
I20260809 19:29:48.611265 12108 sys_catalog.cc:458] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:48.611452 12107 sys_catalog.cc:455] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e39385b3dc1a4ffb95959aad665e89f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e39385b3dc1a4ffb95959aad665e89f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33569 } } }
I20260809 19:29:48.611496 12107 sys_catalog.cc:458] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8 [sys.catalog]: This master's current role is: LEADER
W20260809 19:29:48.611868 12123 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:29:48.611912 12123 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:29:48.612545 12124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:48.612743 12124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:48.614349 12124 catalog_manager.cc:1347] Generated new cluster ID: 1b27945a89e9419d8015c267a426f4d6
I20260809 19:29:48.614439 12124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:48.615759 12110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:48.610093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:57861 (local address 127.0.0.1:33569)
0809 19:29:48.610263 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:48.610275 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:48.610324 (+ 49us) server_negotiation.cc:407] Connection header received
0809 19:29:48.610461 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:48.610468 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:48.610534 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:48.610705 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:48.610998 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:48.612315 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:48.613400 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:48.614026 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:48.614128 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:48.614895 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:48.614898 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:48.614899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:48.615102 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:48.615139 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:48.615153 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:48.615469 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:48.615513 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 19:29:48.615543 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260809 19:29:48.617026 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:48.621192 12124 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:48.621896 12124 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:48.622308 12110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:48.618580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:38131 (local address 127.0.0.1:33569)
0809 19:29:48.618611 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:48.618613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:48.618622 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:29:48.618692 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:48.618693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:48.618703 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:48.618748 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:48.618892 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:48.619910 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:48.621018 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:48.621321 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:48.621386 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:48.622048 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:48.622050 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:48.622050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:48.622054 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:48.622068 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:48.622072 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:48.622130 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:48.622228 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 19:29:48.622239 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:29:48.629616 12124 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e39385b3dc1a4ffb95959aad665e89f8: Generated new TSK 0
I20260809 19:29:48.629797 12124 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:48.633545 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 12030
W20260809 19:29:48.640534 12013 connection.cc:535] client connection to 127.0.0.1:33569 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33569): Connection reset by peer (error 104)
2026-08-09T19:29:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1318 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:48 dist-test-slave-njxd krb5kdc[12136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:48 dist-test-slave-njxd krb5kdc[12136](info): set up 2 sockets
Aug 09 19:29:48 dist-test-slave-njxd krb5kdc[12136](info): commencing operation
krb5kdc: starting...
W20260809 19:29:49.696662 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:49 dist-test-slave-njxd krb5kdc[12136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:29:49.816337 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36079
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:49.896740 12149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:49.896981 12149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:49.897017 12149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:49.898172 12149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:49.898250 12149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:49.898275 12149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:49.898303 12149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:49.898332 12149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:49.899776 12149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36079
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:49.900059 12149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:49.900302 12149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:49.905241 12158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:49.905273 12156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:49.905320 12149 server_base.cc:1034] running on GCE node
W20260809 19:29:49.905274 12155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:49.905718 12149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:49.906011 12149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:49.907178 12149 hybrid_clock.cc:648] HybridClock initialized: now 1786303789907155 us; error 53 us; skew 500 ppm
Aug 09 19:29:49 dist-test-slave-njxd krb5kdc[12136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303789, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:49.909817 12149 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:29:49.910274 12149 webserver.cc:460] Webserver started at http://127.0.0.1:39503/ using document root <none> and password file <none>
I20260809 19:29:49.910449 12149 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:49.910487 12149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:49.910573 12149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:49.911576 12149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "6f3005555c3d45a28cbe7177a7d1940a"
format_stamp: "Formatted at 2026-08-09 19:29:49 on dist-test-slave-njxd"
server_key: "27c48733519be40c8c60581ea070c07d"
server_key_iv: "a19d8fde060fbb4aac8694e82113cb69"
server_key_version: "encryptionkey@0"
I20260809 19:29:49.911870 12149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "6f3005555c3d45a28cbe7177a7d1940a"
format_stamp: "Formatted at 2026-08-09 19:29:49 on dist-test-slave-njxd"
server_key: "27c48733519be40c8c60581ea070c07d"
server_key_iv: "a19d8fde060fbb4aac8694e82113cb69"
server_key_version: "encryptionkey@0"
I20260809 19:29:49.913199 12149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:29:49.913954 12166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:49.914136 12149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:49.914225 12149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "6f3005555c3d45a28cbe7177a7d1940a"
format_stamp: "Formatted at 2026-08-09 19:29:49 on dist-test-slave-njxd"
server_key: "27c48733519be40c8c60581ea070c07d"
server_key_iv: "a19d8fde060fbb4aac8694e82113cb69"
server_key_version: "encryptionkey@0"
I20260809 19:29:49.914290 12149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:49.917531 12149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:49.919340 12149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:49.919463 12149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:49.923379 12149 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36079
I20260809 19:29:49.923432 12222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36079 every 8 connection(s)
I20260809 19:29:49.923722 12149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:49.924760 12223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:49.927093 12223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a: Bootstrap starting.
I20260809 19:29:49.927673 12223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:49.927953 12223 log.cc:824] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:49.928618 12223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a: No bootstrap required, opened a new log
I20260809 19:29:49.930330 12223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3005555c3d45a28cbe7177a7d1940a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } }
I20260809 19:29:49.930485 12223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:49.930534 12223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f3005555c3d45a28cbe7177a7d1940a, State: Initialized, Role: FOLLOWER
I20260809 19:29:49.930699 12223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3005555c3d45a28cbe7177a7d1940a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } }
I20260809 19:29:49.930830 12223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:49.930876 12223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:49.930930 12223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:49.931336 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 12149
I20260809 19:29:49.931432 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:49.931530 12223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3005555c3d45a28cbe7177a7d1940a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } }
I20260809 19:29:49.931671 21831 external_mini_cluster.cc:1428] Setting key 0deead197bb1ce26a64a72348a5aea57
I20260809 19:29:49.931658 12223 leader_election.cc:302] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f3005555c3d45a28cbe7177a7d1940a; no voters:
I20260809 19:29:49.931830 12223 leader_election.cc:288] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:49.931860 12227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:49.931974 12227 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [term 1 LEADER]: Becoming Leader. State: Replica: 6f3005555c3d45a28cbe7177a7d1940a, State: Running, Role: LEADER
I20260809 19:29:49.932057 12227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3005555c3d45a28cbe7177a7d1940a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } }
I20260809 19:29:49.932087 12223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:49.932376 12227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f3005555c3d45a28cbe7177a7d1940a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3005555c3d45a28cbe7177a7d1940a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } } }
I20260809 19:29:49.932482 12227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:49.932650 12227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f3005555c3d45a28cbe7177a7d1940a. Latest consensus state: current_term: 1 leader_uuid: "6f3005555c3d45a28cbe7177a7d1940a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3005555c3d45a28cbe7177a7d1940a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } } }
I20260809 19:29:49.932711 12227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:49.932823 12233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:49.933378 12233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:49.934752 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:49.932373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:52807 (local address 127.0.0.1:36079)
0809 19:29:49.933252 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:49.933260 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:49.933302 (+ 42us) server_negotiation.cc:407] Connection header received
0809 19:29:49.933386 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:49.933392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:49.933414 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:49.933561 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:49.934256 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:49.934260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:49.934260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:49.934264 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:49.934318 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:49.934332 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:49.934413 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:49.934487 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 19:29:49.934532 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":78,"threads_started":1}
I20260809 19:29:49.935539 12233 catalog_manager.cc:1347] Generated new cluster ID: cfee823fe5bc4daf949292be7a6c64f6
I20260809 19:29:49.935611 12233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:49.935737 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:49.938508 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:49.937460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.106:54065 (local address 127.0.0.1:36079)
0809 19:29:49.937498 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:49.937500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:49.937606 (+ 106us) server_negotiation.cc:407] Connection header received
0809 19:29:49.937670 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:49.937671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:49.937675 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:49.937725 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:49.938291 (+ 566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:49.938292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:49.938292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:49.938296 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:49.938310 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:49.938313 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:49.938364 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:49.938423 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 19:29:49.938435 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:29:49.964560 12233 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:49.964775 12233 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:49.967512 12233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f3005555c3d45a28cbe7177a7d1940a: Generated new TSK 0
I20260809 19:29:49.967718 12233 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:49.977208 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 12149
2026-08-09T19:29:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1342 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:50 dist-test-slave-njxd krb5kdc[12256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:50 dist-test-slave-njxd krb5kdc[12256](info): set up 2 sockets
Aug 09 19:29:50 dist-test-slave-njxd krb5kdc[12256](info): commencing operation
krb5kdc: starting...
W20260809 19:29:51.037531 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:51 dist-test-slave-njxd krb5kdc[12256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:29:51.158754 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39363
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:39363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:51.237650 12269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:51.237834 12269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:51.237855 12269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:51.238950 12269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:51.238991 12269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:51.239004 12269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:51.239053 12269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:51.239073 12269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:51.240437 12269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:39363
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:39363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:51.240664 12269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:51.240886 12269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:51.245528 12277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:51.245551 12275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:51.245529 12274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:51.245571 12269 server_base.cc:1034] running on GCE node
I20260809 19:29:51.245998 12269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:51.246315 12269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:51.247478 12269 hybrid_clock.cc:648] HybridClock initialized: now 1786303791247470 us; error 34 us; skew 500 ppm
Aug 09 19:29:51 dist-test-slave-njxd krb5kdc[12256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303791, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:51.252745 12269 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:29:51.253193 12269 webserver.cc:460] Webserver started at http://127.21.81.254:38437/ using document root <none> and password file <none>
I20260809 19:29:51.253412 12269 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:51.253459 12269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:51.253677 12269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:51.254733 12269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "e660f0c8e985485fbb924f0bf090155d"
format_stamp: "Formatted at 2026-08-09 19:29:51 on dist-test-slave-njxd"
server_key: "c5c3976cf35578fa605bb9556c669cdf"
server_key_iv: "7beefe3dff078f4958868514cac1f61b"
server_key_version: "encryptionkey@0"
I20260809 19:29:51.255090 12269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "e660f0c8e985485fbb924f0bf090155d"
format_stamp: "Formatted at 2026-08-09 19:29:51 on dist-test-slave-njxd"
server_key: "c5c3976cf35578fa605bb9556c669cdf"
server_key_iv: "7beefe3dff078f4958868514cac1f61b"
server_key_version: "encryptionkey@0"
I20260809 19:29:51.256518 12269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:29:51.257269 12284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:51.257464 12269 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:51.257558 12269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "e660f0c8e985485fbb924f0bf090155d"
format_stamp: "Formatted at 2026-08-09 19:29:51 on dist-test-slave-njxd"
server_key: "c5c3976cf35578fa605bb9556c669cdf"
server_key_iv: "7beefe3dff078f4958868514cac1f61b"
server_key_version: "encryptionkey@0"
I20260809 19:29:51.257673 12269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:51.270757 12269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:51.272943 12269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:51.273051 12269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:51.277177 12269 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:39363
I20260809 19:29:51.277235 12342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:39363 every 8 connection(s)
I20260809 19:29:51.277575 12269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:51.278512 12343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:51.281071 12343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d: Bootstrap starting.
I20260809 19:29:51.281816 12343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:51.282117 12343 log.cc:824] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:51.282837 12343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d: No bootstrap required, opened a new log
I20260809 19:29:51.283365 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 12269
I20260809 19:29:51.283490 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:51.283671 21831 external_mini_cluster.cc:1428] Setting key efe9bd46d97f52d04a71937f464cb6f5
I20260809 19:29:51.285530 12343 raft_consensus.cc:348] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e660f0c8e985485fbb924f0bf090155d" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39363 } }
I20260809 19:29:51.285730 12343 raft_consensus.cc:374] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:51.285806 12343 raft_consensus.cc:729] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e660f0c8e985485fbb924f0bf090155d, State: Initialized, Role: FOLLOWER
I20260809 19:29:51.285940 12343 consensus_queue.cc:260] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e660f0c8e985485fbb924f0bf090155d" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39363 } }
I20260809 19:29:51.286021 12343 raft_consensus.cc:388] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:51.286044 12343 raft_consensus.cc:482] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:51.286123 12343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:51.286765 12343 raft_consensus.cc:504] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e660f0c8e985485fbb924f0bf090155d" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39363 } }
I20260809 19:29:51.286868 12343 leader_election.cc:302] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e660f0c8e985485fbb924f0bf090155d; no voters:
I20260809 19:29:51.287016 12343 leader_election.cc:288] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:51.287094 12349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:51.287271 12343 sys_catalog.cc:564] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:51.287276 12349 raft_consensus.cc:686] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [term 1 LEADER]: Becoming Leader. State: Replica: e660f0c8e985485fbb924f0bf090155d, State: Running, Role: LEADER
I20260809 19:29:51.287499 12349 consensus_queue.cc:237] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e660f0c8e985485fbb924f0bf090155d" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39363 } }
I20260809 19:29:51.287897 12350 sys_catalog.cc:455] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e660f0c8e985485fbb924f0bf090155d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e660f0c8e985485fbb924f0bf090155d" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39363 } } }
I20260809 19:29:51.287992 12350 sys_catalog.cc:458] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:51.288206 12351 sys_catalog.cc:455] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e660f0c8e985485fbb924f0bf090155d. Latest consensus state: current_term: 1 leader_uuid: "e660f0c8e985485fbb924f0bf090155d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e660f0c8e985485fbb924f0bf090155d" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 39363 } } }
I20260809 19:29:51.288512 12351 sys_catalog.cc:458] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d [sys.catalog]: This master's current role is: LEADER
W20260809 19:29:51.289057 12368 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:29:51.289112 12368 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:29:51.289201 12358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:51.289395 12358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:51.291107 12358 catalog_manager.cc:1347] Generated new cluster ID: 051441d127d143eca7927ae65819c9d3
I20260809 19:29:51.291163 12358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:29:51 dist-test-slave-njxd krb5kdc[12256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:29:51.295837 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:51.284213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53881 (local address 127.21.81.254:39363)
0809 19:29:51.284436 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:51.284445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:51.284479 (+ 34us) server_negotiation.cc:407] Connection header received
0809 19:29:51.285484 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:51.285493 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:51.285557 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:51.285771 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:51.286586 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:51.288043 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:51.289230 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:51.289811 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:51.289928 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:51.292552 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:51.292556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:51.292556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:51.292567 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:51.294282 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:51.294691 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:51.294699 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:51.294699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:51.294778 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:51.295009 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:51.295010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:51.295010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:51.295239 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:51.295415 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:51.295533 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 19:29:51.295637 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":99,"threads_started":1}
I20260809 19:29:51.296774 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:51.302898 12358 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:51.303591 12358 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:51.308382 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:51.301895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53197 (local address 127.21.81.254:39363)
0809 19:29:51.301937 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:51.301939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:51.302128 (+ 189us) server_negotiation.cc:407] Connection header received
0809 19:29:51.302204 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:51.302204 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:51.302217 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:51.302275 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:51.303024 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:51.303523 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:51.304261 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:51.304488 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:51.304562 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:51.305802 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:51.305803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:51.305803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:51.305811 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:51.307215 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:51.307582 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:51.307583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:51.307583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:51.307646 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:51.307861 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:51.307862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:51.307862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:51.307967 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:51.308073 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:51.308215 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 19:29:51.308277 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:29:51.310771 12358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e660f0c8e985485fbb924f0bf090155d: Generated new TSK 0
I20260809 19:29:51.310932 12358 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:51.316102 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 12269
W20260809 19:29:51.322499 12254 connection.cc:535] client connection to 127.21.81.254:39363 recv error: Network error: failed to read from TLS socket (remote: 127.21.81.254:39363): Connection reset by peer (error 104)
2026-08-09T19:29:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:51 dist-test-slave-njxd krb5kdc[12379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:51 dist-test-slave-njxd krb5kdc[12379](info): set up 2 sockets
Aug 09 19:29:51 dist-test-slave-njxd krb5kdc[12379](info): commencing operation
krb5kdc: starting...
W20260809 19:29:52.373172 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:52 dist-test-slave-njxd krb5kdc[12379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:29:52.494652 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:43407
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:44369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:43407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:52.577912 12392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:52.578138 12392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:52.578163 12392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:52.579411 12392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:52.579457 12392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:52.579473 12392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:52.579485 12392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:52.579497 12392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:52.581020 12392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:44369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:43407
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:43407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:52.581228 12392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:52.581408 12392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:52.586274 12399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:52.586294 12400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:52.586274 12402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:52.586522 12392 server_base.cc:1034] running on GCE node
I20260809 19:29:52.586766 12392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:52.587062 12392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:52.588238 12392 hybrid_clock.cc:648] HybridClock initialized: now 1786303792588225 us; error 49 us; skew 500 ppm
Aug 09 19:29:52 dist-test-slave-njxd krb5kdc[12379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303792, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:52.590734 12392 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:29:52.591177 12392 webserver.cc:460] Webserver started at http://127.21.81.254:33193/ using document root <none> and password file <none>
I20260809 19:29:52.591423 12392 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:52.591492 12392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:52.591660 12392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:52.592753 12392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "648b71c84a3b40169ca7f95de808be51"
format_stamp: "Formatted at 2026-08-09 19:29:52 on dist-test-slave-njxd"
server_key: "ed017a09bd3ce4dbb0d0979ed5706ba2"
server_key_iv: "ddfce2b49e499143ec347236e9717b98"
server_key_version: "encryptionkey@0"
I20260809 19:29:52.593118 12392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "648b71c84a3b40169ca7f95de808be51"
format_stamp: "Formatted at 2026-08-09 19:29:52 on dist-test-slave-njxd"
server_key: "ed017a09bd3ce4dbb0d0979ed5706ba2"
server_key_iv: "ddfce2b49e499143ec347236e9717b98"
server_key_version: "encryptionkey@0"
I20260809 19:29:52.594578 12392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:29:52.595316 12409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:52.595486 12392 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:52.595594 12392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "648b71c84a3b40169ca7f95de808be51"
format_stamp: "Formatted at 2026-08-09 19:29:52 on dist-test-slave-njxd"
server_key: "ed017a09bd3ce4dbb0d0979ed5706ba2"
server_key_iv: "ddfce2b49e499143ec347236e9717b98"
server_key_version: "encryptionkey@0"
I20260809 19:29:52.595669 12392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:52.611852 12392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:52.614055 12392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:52.614176 12392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:52.618278 12392 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:43407
I20260809 19:29:52.618332 12466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:43407 every 8 connection(s)
I20260809 19:29:52.618696 12392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:52.619688 12467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:52.619750 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 12392
I20260809 19:29:52.619846 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:52.620045 21831 external_mini_cluster.cc:1428] Setting key c72b50239716cef19afabdb4ff5a4188
I20260809 19:29:52.622758 12467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51: Bootstrap starting.
I20260809 19:29:52.623370 12467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:52.623673 12467 log.cc:824] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:52.624413 12467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51: No bootstrap required, opened a new log
I20260809 19:29:52.625402 12470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:52.620617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60967 (local address 127.21.81.254:43407)
0809 19:29:52.620832 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:52.620848 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:52.620874 (+ 26us) server_negotiation.cc:407] Connection header received
0809 19:29:52.621290 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:52.621296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:52.621342 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:52.621489 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:52.621736 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:52.622607 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:52.623481 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:52.623936 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:52.624054 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:52.624822 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:52.624826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:52.624827 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:52.624991 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:52.625020 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:52.625030 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:52.625123 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:52.625202 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 19:29:52.625232 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":81,"threads_started":1}
I20260809 19:29:52.626622 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:52.626573 12467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648b71c84a3b40169ca7f95de808be51" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43407 } }
I20260809 19:29:52.626751 12467 raft_consensus.cc:374] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:52.626796 12467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648b71c84a3b40169ca7f95de808be51, State: Initialized, Role: FOLLOWER
I20260809 19:29:52.626920 12467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648b71c84a3b40169ca7f95de808be51" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43407 } }
I20260809 19:29:52.626988 12467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:52.627010 12467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:52.627074 12467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:52.627617 12467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648b71c84a3b40169ca7f95de808be51" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43407 } }
I20260809 19:29:52.627734 12467 leader_election.cc:302] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 648b71c84a3b40169ca7f95de808be51; no voters:
I20260809 19:29:52.627911 12467 leader_election.cc:288] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:52.627940 12477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:52.628082 12477 raft_consensus.cc:686] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [term 1 LEADER]: Becoming Leader. State: Replica: 648b71c84a3b40169ca7f95de808be51, State: Running, Role: LEADER
I20260809 19:29:52.628206 12467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:52.628223 12477 consensus_queue.cc:237] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648b71c84a3b40169ca7f95de808be51" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43407 } }
I20260809 19:29:52.628543 12477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "648b71c84a3b40169ca7f95de808be51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648b71c84a3b40169ca7f95de808be51" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43407 } } }
I20260809 19:29:52.628621 12477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:52.628801 12477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 648b71c84a3b40169ca7f95de808be51. Latest consensus state: current_term: 1 leader_uuid: "648b71c84a3b40169ca7f95de808be51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "648b71c84a3b40169ca7f95de808be51" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 43407 } } }
I20260809 19:29:52.628875 12477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:52.629228 12484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:52.629834 12484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:52.631516 12484 catalog_manager.cc:1347] Generated new cluster ID: 95657f0e639f4022916c400183825927
I20260809 19:29:52.631568 12484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:52.633101 12470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:52.628127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35203 (local address 127.21.81.254:43407)
0809 19:29:52.628164 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:52.628165 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:52.629774 (+ 1609us) server_negotiation.cc:407] Connection header received
0809 19:29:52.629900 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:52.629901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:52.629911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:52.629983 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:52.630167 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:52.630773 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:52.631776 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:52.632090 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:52.632168 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:52.632858 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:52.632860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:52.632860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:52.632864 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:52.632878 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:52.632883 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:52.632958 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:52.633025 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 19:29:52.633035 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:29:52.671433 12484 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:52.672209 12484 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:52.675828 12484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 648b71c84a3b40169ca7f95de808be51: Generated new TSK 0
I20260809 19:29:52.676005 12484 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:52.679726 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 12392
W20260809 19:29:52.686569 12377 connection.cc:535] client connection to 127.21.81.254:43407 recv error: Network error: failed to read from TLS socket (remote: 127.21.81.254:43407): Connection reset by peer (error 104)
2026-08-09T19:29:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1364 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:52 dist-test-slave-njxd krb5kdc[12500](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:52 dist-test-slave-njxd krb5kdc[12500](info): set up 2 sockets
Aug 09 19:29:52 dist-test-slave-njxd krb5kdc[12500](info): commencing operation
krb5kdc: starting...
W20260809 19:29:53.743026 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:53 dist-test-slave-njxd krb5kdc[12500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:29:53.864637 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:45097
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:38217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:45097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:53.947464 12518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:53.947682 12518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:53.947705 12518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:53.948894 12518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:53.948935 12518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:53.948949 12518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:53.948961 12518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:53.949010 12518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:53.950572 12518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:38217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:45097
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:45097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:53.950819 12518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:53.951054 12518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:53.956097 12523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:53.956118 12524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:53.956141 12518 server_base.cc:1034] running on GCE node
W20260809 19:29:53.956077 12526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:53.956575 12518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:53.956862 12518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:53.958003 12518 hybrid_clock.cc:648] HybridClock initialized: now 1786303793957987 us; error 33 us; skew 500 ppm
Aug 09 19:29:53 dist-test-slave-njxd krb5kdc[12500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303793, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:53.960569 12518 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:29:53.961047 12518 webserver.cc:460] Webserver started at http://127.21.81.254:36841/ using document root <none> and password file <none>
I20260809 19:29:53.961261 12518 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:53.961302 12518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:53.961390 12518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:53.962488 12518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "2c413089265e4694b26db94d47c20331"
format_stamp: "Formatted at 2026-08-09 19:29:53 on dist-test-slave-njxd"
server_key: "5d32fc5d944ad0325e8be2635761d727"
server_key_iv: "bd5bbb14cbdb681ca5dca26bc93d292e"
server_key_version: "encryptionkey@0"
I20260809 19:29:53.962809 12518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "2c413089265e4694b26db94d47c20331"
format_stamp: "Formatted at 2026-08-09 19:29:53 on dist-test-slave-njxd"
server_key: "5d32fc5d944ad0325e8be2635761d727"
server_key_iv: "bd5bbb14cbdb681ca5dca26bc93d292e"
server_key_version: "encryptionkey@0"
I20260809 19:29:53.964121 12518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:29:53.964838 12533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:53.965016 12518 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:53.965106 12518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "2c413089265e4694b26db94d47c20331"
format_stamp: "Formatted at 2026-08-09 19:29:53 on dist-test-slave-njxd"
server_key: "5d32fc5d944ad0325e8be2635761d727"
server_key_iv: "bd5bbb14cbdb681ca5dca26bc93d292e"
server_key_version: "encryptionkey@0"
I20260809 19:29:53.965163 12518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:53.968564 12518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:53.971261 12518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:53.971417 12518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:53.975380 12518 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:45097
I20260809 19:29:53.975416 12588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:45097 every 8 connection(s)
I20260809 19:29:53.975756 12518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:53.976788 12589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:53.979175 12589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331: Bootstrap starting.
I20260809 19:29:53.979755 12589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:53.980041 12589 log.cc:824] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:53.980268 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 12518
I20260809 19:29:53.980389 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:53.980561 21831 external_mini_cluster.cc:1428] Setting key 7718d677be60fa1874a1c8497d4bfd0d
I20260809 19:29:53.980872 12589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331: No bootstrap required, opened a new log
I20260809 19:29:53.983214 12589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c413089265e4694b26db94d47c20331" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45097 } }
I20260809 19:29:53.983430 12589 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:53.983496 12589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c413089265e4694b26db94d47c20331, State: Initialized, Role: FOLLOWER
I20260809 19:29:53.983631 12589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c413089265e4694b26db94d47c20331" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45097 } }
I20260809 19:29:53.983740 12589 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:53.983778 12589 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:53.983799 12593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:53.981134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50605 (local address 127.21.81.254:45097)
0809 19:29:53.981357 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:53.981367 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:53.981403 (+ 36us) server_negotiation.cc:407] Connection header received
0809 19:29:53.982289 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:53.982299 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:53.982334 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:53.982483 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:53.983137 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:53.983140 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:53.983141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:53.983145 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:53.983192 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:53.983207 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:53.983281 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:53.983338 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 19:29:53.983379 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260809 19:29:53.983834 12589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:53.984442 12589 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c413089265e4694b26db94d47c20331" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45097 } }
I20260809 19:29:53.984572 12589 leader_election.cc:302] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c413089265e4694b26db94d47c20331; no voters:
I20260809 19:29:53.984860 12589 leader_election.cc:288] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:53.984887 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:53.985090 12595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:53.985198 12589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:53.985249 12595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [term 1 LEADER]: Becoming Leader. State: Replica: 2c413089265e4694b26db94d47c20331, State: Running, Role: LEADER
I20260809 19:29:53.985411 12595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c413089265e4694b26db94d47c20331" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45097 } }
I20260809 19:29:53.985767 12596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c413089265e4694b26db94d47c20331" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c413089265e4694b26db94d47c20331" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45097 } } }
I20260809 19:29:53.985764 12595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c413089265e4694b26db94d47c20331. Latest consensus state: current_term: 1 leader_uuid: "2c413089265e4694b26db94d47c20331" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c413089265e4694b26db94d47c20331" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45097 } } }
I20260809 19:29:53.985950 12596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:53.986050 12595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:53.986336 12609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:53.986912 12609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:53.988667 12609 catalog_manager.cc:1347] Generated new cluster ID: 96db02f4874b45359ac6c3d4338d1b16
I20260809 19:29:53.988719 12609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:53.988894 12593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:53.987627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51905 (local address 127.21.81.254:45097)
0809 19:29:53.987666 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:53.987668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:53.987870 (+ 202us) server_negotiation.cc:407] Connection header received
0809 19:29:53.987918 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:53.987919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:53.987936 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:53.988000 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:53.988606 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:53.988607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:53.988608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:53.988612 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:53.988627 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:53.988632 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:53.988688 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:53.988825 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 19:29:53.988837 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:29:54.004814 12609 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:54.005012 12609 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:54.008638 12609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c413089265e4694b26db94d47c20331: Generated new TSK 0
I20260809 19:29:54.008838 12609 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:54.014155 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 12518
2026-08-09T19:29:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:54 dist-test-slave-njxd krb5kdc[12624](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:54 dist-test-slave-njxd krb5kdc[12624](info): set up 2 sockets
Aug 09 19:29:54 dist-test-slave-njxd krb5kdc[12624](info): commencing operation
krb5kdc: starting...
W20260809 19:29:55.070361 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:55 dist-test-slave-njxd krb5kdc[12624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:29:55.192358 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:35685
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:45719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:35685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:55.273298 12637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:55.273520 12637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:55.273545 12637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:55.274797 12637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:55.274847 12637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:55.274863 12637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:55.274875 12637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:55.274927 12637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:55.276507 12637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:45719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:35685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:35685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:55.276784 12637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:55.277038 12637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:55.282047 12644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:55.282122 12643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:55.282047 12647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:55.282114 12637 server_base.cc:1034] running on GCE node
I20260809 19:29:55.282544 12637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:55.282846 12637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:55.283989 12637 hybrid_clock.cc:648] HybridClock initialized: now 1786303795283976 us; error 30 us; skew 500 ppm
Aug 09 19:29:55 dist-test-slave-njxd krb5kdc[12624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303795, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:55.286531 12637 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:29:55.287045 12637 webserver.cc:460] Webserver started at http://127.21.81.254:40081/ using document root <none> and password file <none>
I20260809 19:29:55.287259 12637 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:55.287302 12637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:55.287391 12637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:55.288405 12637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "6907d0600bd140f798f47390c306a2bc"
format_stamp: "Formatted at 2026-08-09 19:29:55 on dist-test-slave-njxd"
server_key: "53fcc75651ec6a4185bb2102ed5e6248"
server_key_iv: "a191e9e1dabd63eac0fca71fcbc9dbbb"
server_key_version: "encryptionkey@0"
I20260809 19:29:55.288700 12637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "6907d0600bd140f798f47390c306a2bc"
format_stamp: "Formatted at 2026-08-09 19:29:55 on dist-test-slave-njxd"
server_key: "53fcc75651ec6a4185bb2102ed5e6248"
server_key_iv: "a191e9e1dabd63eac0fca71fcbc9dbbb"
server_key_version: "encryptionkey@0"
I20260809 19:29:55.290097 12637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:29:55.290805 12655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:55.291003 12637 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:29:55.291096 12637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "6907d0600bd140f798f47390c306a2bc"
format_stamp: "Formatted at 2026-08-09 19:29:55 on dist-test-slave-njxd"
server_key: "53fcc75651ec6a4185bb2102ed5e6248"
server_key_iv: "a191e9e1dabd63eac0fca71fcbc9dbbb"
server_key_version: "encryptionkey@0"
I20260809 19:29:55.291154 12637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:55.307331 12637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:55.309130 12637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:55.309242 12637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:55.313227 12637 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:35685
I20260809 19:29:55.313282 12713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:35685 every 8 connection(s)
I20260809 19:29:55.313681 12637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:55.314585 12714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:55.316874 12714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc: Bootstrap starting.
I20260809 19:29:55.317158 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 12637
I20260809 19:29:55.317283 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:55.317467 21831 external_mini_cluster.cc:1428] Setting key 79d6ed7c7bc6406baf910b28c7744862
I20260809 19:29:55.317667 12714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:55.318032 12714 log.cc:824] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:55.318735 12714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc: No bootstrap required, opened a new log
I20260809 19:29:55.320533 12714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6907d0600bd140f798f47390c306a2bc" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 35685 } }
I20260809 19:29:55.320686 12714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:55.320735 12714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6907d0600bd140f798f47390c306a2bc, State: Initialized, Role: FOLLOWER
I20260809 19:29:55.320858 12714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6907d0600bd140f798f47390c306a2bc" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 35685 } }
I20260809 19:29:55.320955 12714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:55.320999 12714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:55.321045 12714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:55.321552 12714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6907d0600bd140f798f47390c306a2bc" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 35685 } }
I20260809 19:29:55.321697 12714 leader_election.cc:302] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6907d0600bd140f798f47390c306a2bc; no voters:
I20260809 19:29:55.321902 12714 leader_election.cc:288] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:55.321925 12720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:55.322152 12720 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [term 1 LEADER]: Becoming Leader. State: Replica: 6907d0600bd140f798f47390c306a2bc, State: Running, Role: LEADER
I20260809 19:29:55.322260 12714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:55.322633 12720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6907d0600bd140f798f47390c306a2bc" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 35685 } }
Aug 09 19:29:55 dist-test-slave-njxd krb5kdc[12624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
W20260809 19:29:55.324458 12736 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 19:29:55.324455 12721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6907d0600bd140f798f47390c306a2bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6907d0600bd140f798f47390c306a2bc" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 35685 } } }
W20260809 19:29:55.324632 12736 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:29:55.324651 12721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:55.324211 12722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6907d0600bd140f798f47390c306a2bc. Latest consensus state: current_term: 1 leader_uuid: "6907d0600bd140f798f47390c306a2bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6907d0600bd140f798f47390c306a2bc" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 35685 } } }
I20260809 19:29:55.324849 12722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:55.324991 12738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:55.325186 12738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:55.326831 12738 catalog_manager.cc:1347] Generated new cluster ID: 4dbaf79a750f42049d37f6b47ec51332
I20260809 19:29:55.326927 12738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:55.328424 12717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:55.318039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41105 (local address 127.21.81.254:35685)
0809 19:29:55.318232 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:55.318245 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:55.318270 (+ 25us) server_negotiation.cc:407] Connection header received
0809 19:29:55.318676 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:55.318681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:55.318724 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:55.318933 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:55.319550 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:55.320409 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:55.321318 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:55.321856 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:55.322015 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:55.324828 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:55.324832 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:55.324833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:55.324845 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:55.326812 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:55.327237 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:55.327240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:55.327240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:55.327330 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:55.327537 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:55.327538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:55.327539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:55.327792 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:55.327964 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:55.328069 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 19:29:55.328219 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":69,"threads_started":1}
I20260809 19:29:55.329324 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:55.337368 12717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:55.330916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58547 (local address 127.21.81.254:35685)
0809 19:29:55.330951 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:55.330953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:55.331099 (+ 146us) server_negotiation.cc:407] Connection header received
0809 19:29:55.331161 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:55.331161 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:55.331172 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:55.331219 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:55.331720 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:55.332231 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:55.332936 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:55.333152 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:55.333238 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:55.334449 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:55.334450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:55.334451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:55.334473 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:55.336030 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:55.336460 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:55.336462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:55.336462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:55.336524 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:55.336750 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:55.336751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:55.336752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:55.336897 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:55.336989 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:55.337177 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 19:29:55.337257 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:29:55.395197 12738 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:55.395936 12738 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:55.402901 12738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6907d0600bd140f798f47390c306a2bc: Generated new TSK 0
I20260809 19:29:55.403098 12738 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:55.409258 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 12637
2026-08-09T19:29:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1394 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:55 dist-test-slave-njxd krb5kdc[12751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:55 dist-test-slave-njxd krb5kdc[12751](info): set up 2 sockets
Aug 09 19:29:55 dist-test-slave-njxd krb5kdc[12751](info): commencing operation
krb5kdc: starting...
W20260809 19:29:56.500833 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:56 dist-test-slave-njxd krb5kdc[12751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:29:56.620234 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:36051
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:36633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:36051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:56.701534 12765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:56.701748 12765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:56.701771 12765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:56.702858 12765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:56.702891 12765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:56.702905 12765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:56.702917 12765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:56.702929 12765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:56.704300 12765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:36633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:36051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:36051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:56.704493 12765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:56.704730 12765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:56.709461 12773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:56.709461 12770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:56.709512 12771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:56.709771 12765 server_base.cc:1034] running on GCE node
I20260809 19:29:56.710003 12765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:56.710337 12765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:56.711498 12765 hybrid_clock.cc:648] HybridClock initialized: now 1786303796711482 us; error 44 us; skew 500 ppm
Aug 09 19:29:56 dist-test-slave-njxd krb5kdc[12751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303796, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:56.714038 12765 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:29:56.714510 12765 webserver.cc:460] Webserver started at http://127.21.81.254:42519/ using document root <none> and password file <none>
I20260809 19:29:56.714769 12765 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:56.714833 12765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:56.714928 12765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:56.715952 12765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "d0dca06dcc344f5fbcf4fd0fb5295774"
format_stamp: "Formatted at 2026-08-09 19:29:56 on dist-test-slave-njxd"
server_key: "e198b494554787a00743139265dd9646"
server_key_iv: "662a142cabe2a8893ac088894bc85e7f"
server_key_version: "encryptionkey@0"
I20260809 19:29:56.716245 12765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "d0dca06dcc344f5fbcf4fd0fb5295774"
format_stamp: "Formatted at 2026-08-09 19:29:56 on dist-test-slave-njxd"
server_key: "e198b494554787a00743139265dd9646"
server_key_iv: "662a142cabe2a8893ac088894bc85e7f"
server_key_version: "encryptionkey@0"
I20260809 19:29:56.717535 12765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:29:56.718365 12780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:56.718549 12765 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:29:56.718643 12765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "d0dca06dcc344f5fbcf4fd0fb5295774"
format_stamp: "Formatted at 2026-08-09 19:29:56 on dist-test-slave-njxd"
server_key: "e198b494554787a00743139265dd9646"
server_key_iv: "662a142cabe2a8893ac088894bc85e7f"
server_key_version: "encryptionkey@0"
I20260809 19:29:56.718706 12765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:56.737022 12765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:56.739404 12765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:56.739537 12765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:56.743495 12765 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:36051
I20260809 19:29:56.743558 12840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:36051 every 8 connection(s)
I20260809 19:29:56.743918 12765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:56.744838 12842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:56.745374 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 12765
I20260809 19:29:56.745503 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:56.745693 21831 external_mini_cluster.cc:1428] Setting key cbb29ebe7f6dad8a2d6939b84ff7bc6c
I20260809 19:29:56.747622 12842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774: Bootstrap starting.
I20260809 19:29:56.748243 12842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:56.748545 12842 log.cc:824] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:56.749303 12842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774: No bootstrap required, opened a new log
I20260809 19:29:56.751286 12846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:56.746272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32987 (local address 127.21.81.254:36051)
0809 19:29:56.746472 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:56.746479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:56.746505 (+ 26us) server_negotiation.cc:407] Connection header received
0809 19:29:56.746914 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:56.746920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:56.746966 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:56.747117 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:56.747380 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:56.748338 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:56.749373 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:56.749852 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:56.749971 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:56.750674 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:56.750678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:56.750679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:56.750853 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:56.750882 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:56.750892 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:56.750968 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:56.751044 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 19:29:56.751109 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
I20260809 19:29:56.751217 12842 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0dca06dcc344f5fbcf4fd0fb5295774" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 36051 } }
I20260809 19:29:56.751398 12842 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:56.751451 12842 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0dca06dcc344f5fbcf4fd0fb5295774, State: Initialized, Role: FOLLOWER
I20260809 19:29:56.751561 12842 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0dca06dcc344f5fbcf4fd0fb5295774" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 36051 } }
I20260809 19:29:56.751658 12842 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:56.751710 12842 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:56.751782 12842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:56.752211 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:56.752388 12842 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0dca06dcc344f5fbcf4fd0fb5295774" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 36051 } }
I20260809 19:29:56.752486 12842 leader_election.cc:302] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0dca06dcc344f5fbcf4fd0fb5295774; no voters:
I20260809 19:29:56.752638 12842 leader_election.cc:288] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:56.752708 12848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:56.752866 12848 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [term 1 LEADER]: Becoming Leader. State: Replica: d0dca06dcc344f5fbcf4fd0fb5295774, State: Running, Role: LEADER
I20260809 19:29:56.752914 12842 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:56.753010 12848 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0dca06dcc344f5fbcf4fd0fb5295774" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 36051 } }
I20260809 19:29:56.754444 12849 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0dca06dcc344f5fbcf4fd0fb5295774. Latest consensus state: current_term: 1 leader_uuid: "d0dca06dcc344f5fbcf4fd0fb5295774" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0dca06dcc344f5fbcf4fd0fb5295774" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 36051 } } }
I20260809 19:29:56.754698 12850 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0dca06dcc344f5fbcf4fd0fb5295774" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0dca06dcc344f5fbcf4fd0fb5295774" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 36051 } } }
I20260809 19:29:56.754525 12849 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:56.754810 12850 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:56.755110 12868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:56.755610 12868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:56.757354 12846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:56.753993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46171 (local address 127.21.81.254:36051)
0809 19:29:56.754030 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:56.754032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:56.754063 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:29:56.754081 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:56.754082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:56.754090 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:56.754138 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:56.754326 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:56.755064 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:56.756044 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:56.756335 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:56.756401 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:56.757045 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:56.757047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:56.757047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:56.757052 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:56.757066 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:56.757070 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:56.757131 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:56.757238 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 19:29:56.757262 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:29:56.757366 12868 catalog_manager.cc:1347] Generated new cluster ID: 0635729d98994f6abc448a78cbade7f2
I20260809 19:29:56.757441 12868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:56.771467 12868 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:56.772265 12868 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:56.778016 12868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0dca06dcc344f5fbcf4fd0fb5295774: Generated new TSK 0
I20260809 19:29:56.778236 12868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:56.784986 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 12765
2026-08-09T19:29:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1375 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:56 dist-test-slave-njxd krb5kdc[12876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:56 dist-test-slave-njxd krb5kdc[12876](info): set up 2 sockets
Aug 09 19:29:56 dist-test-slave-njxd krb5kdc[12876](info): commencing operation
krb5kdc: starting...
W20260809 19:29:57.846356 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:57 dist-test-slave-njxd krb5kdc[12876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:29:57.966542 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:34109
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:33701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:34109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:58.047447 12890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:58.047679 12890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:58.047715 12890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:58.048805 12890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:58.048871 12890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:58.048897 12890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:58.048928 12890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:58.048954 12890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:58.050534 12890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:33701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:34109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:34109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:58.050827 12890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:58.051050 12890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:58.055801 12899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:58.055794 12897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:58.055831 12896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:58.055965 12890 server_base.cc:1034] running on GCE node
I20260809 19:29:58.056321 12890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:58.056607 12890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:58.057758 12890 hybrid_clock.cc:648] HybridClock initialized: now 1786303798057746 us; error 28 us; skew 500 ppm
Aug 09 19:29:58 dist-test-slave-njxd krb5kdc[12876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303798, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:58.060336 12890 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:29:58.060798 12890 webserver.cc:460] Webserver started at http://127.21.81.254:37115/ using document root <none> and password file <none>
I20260809 19:29:58.060997 12890 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:58.061038 12890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:58.061128 12890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:58.062197 12890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "42950ece1b0543f4a475c049c686242e"
format_stamp: "Formatted at 2026-08-09 19:29:58 on dist-test-slave-njxd"
server_key: "51b5e8d312a6de7666c723e75b0a396e"
server_key_iv: "a38fac50c0fe8abbd467f2b758b903ff"
server_key_version: "encryptionkey@0"
I20260809 19:29:58.062503 12890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "42950ece1b0543f4a475c049c686242e"
format_stamp: "Formatted at 2026-08-09 19:29:58 on dist-test-slave-njxd"
server_key: "51b5e8d312a6de7666c723e75b0a396e"
server_key_iv: "a38fac50c0fe8abbd467f2b758b903ff"
server_key_version: "encryptionkey@0"
I20260809 19:29:58.063774 12890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:29:58.064460 12907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:58.064632 12890 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:58.064750 12890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "42950ece1b0543f4a475c049c686242e"
format_stamp: "Formatted at 2026-08-09 19:29:58 on dist-test-slave-njxd"
server_key: "51b5e8d312a6de7666c723e75b0a396e"
server_key_iv: "a38fac50c0fe8abbd467f2b758b903ff"
server_key_version: "encryptionkey@0"
I20260809 19:29:58.064826 12890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:58.068209 12890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:58.070742 12890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:58.070912 12890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:58.075106 12890 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:34109
I20260809 19:29:58.075136 12965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:34109 every 8 connection(s)
I20260809 19:29:58.075474 12890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:58.076452 12966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:58.078900 12966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e: Bootstrap starting.
I20260809 19:29:58.079437 12966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:58.079669 12966 log.cc:824] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:58.080380 12966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e: No bootstrap required, opened a new log
I20260809 19:29:58.081801 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 12890
I20260809 19:29:58.081924 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:58.082101 21831 external_mini_cluster.cc:1428] Setting key 7b9fc2f9388cf45c4ced09cd71201344
I20260809 19:29:58.082314 12966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42950ece1b0543f4a475c049c686242e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34109 } }
I20260809 19:29:58.082489 12966 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:29:58.082526 12966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42950ece1b0543f4a475c049c686242e, State: Initialized, Role: FOLLOWER
I20260809 19:29:58.082718 12966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42950ece1b0543f4a475c049c686242e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34109 } }
I20260809 19:29:58.082818 12966 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:29:58.082863 12966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:29:58.082909 12966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:58.083995 12966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42950ece1b0543f4a475c049c686242e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34109 } }
I20260809 19:29:58.084112 12966 leader_election.cc:302] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42950ece1b0543f4a475c049c686242e; no voters:
I20260809 19:29:58.084283 12966 leader_election.cc:288] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:29:58.084414 12972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:58.084499 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:58.082613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42225 (local address 127.21.81.254:34109)
0809 19:29:58.082837 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:58.082847 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:58.082882 (+ 35us) server_negotiation.cc:407] Connection header received
0809 19:29:58.083037 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:58.083043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:58.083066 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:58.083202 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:58.083965 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:58.083968 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:58.083969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:58.083973 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:58.084021 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:58.084031 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:58.084114 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:58.084234 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 19:29:58.084277 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":111,"threads_started":1}
I20260809 19:29:58.084574 12966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:58.084574 12972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [term 1 LEADER]: Becoming Leader. State: Replica: 42950ece1b0543f4a475c049c686242e, State: Running, Role: LEADER
I20260809 19:29:58.084789 12972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42950ece1b0543f4a475c049c686242e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34109 } }
I20260809 19:29:58.085129 12972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42950ece1b0543f4a475c049c686242e. Latest consensus state: current_term: 1 leader_uuid: "42950ece1b0543f4a475c049c686242e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42950ece1b0543f4a475c049c686242e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34109 } } }
I20260809 19:29:58.085255 12972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:58.085145 12973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42950ece1b0543f4a475c049c686242e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42950ece1b0543f4a475c049c686242e" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 34109 } } }
I20260809 19:29:58.085384 12973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:58.085462 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:58.085533 12981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:58.086108 12981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:58.088069 12981 catalog_manager.cc:1347] Generated new cluster ID: 048d1de434fe466b9405c8b70da57f26
I20260809 19:29:58.088171 12981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:58.088292 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:58.087120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56371 (local address 127.21.81.254:34109)
0809 19:29:58.087156 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:58.087158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:58.087273 (+ 115us) server_negotiation.cc:407] Connection header received
0809 19:29:58.087324 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:58.087325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:58.087343 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:58.087406 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:58.088048 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:58.088049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:58.088050 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:29:58.088054 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:58.088071 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:29:58.088075 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:58.088132 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:58.088212 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 19:29:58.088227 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:29:58.098491 12981 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:58.098664 12981 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:58.103802 12981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42950ece1b0543f4a475c049c686242e: Generated new TSK 0
I20260809 19:29:58.103998 12981 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:58.110149 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 12890
2026-08-09T19:29:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1325 ms)
[----------] 18 tests from AuthTokenIssuingTest (24323 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:29:58 dist-test-slave-njxd krb5kdc[12999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:29:58 dist-test-slave-njxd krb5kdc[12999](info): set up 2 sockets
Aug 09 19:29:58 dist-test-slave-njxd krb5kdc[12999](info): commencing operation
krb5kdc: starting...
W20260809 19:29:59.175051 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:29:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:29:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:29:59.298578 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:32885
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:32885,127.21.81.253:37601,127.21.81.252:42557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:29:59.381160 13013 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:29:59.381486 13013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:59.381520 13013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:59.381561 13013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:59.382750 13013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:59.382813 13013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:59.382839 13013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:59.382870 13013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:59.382900 13013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:59.384399 13013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:32885,127.21.81.253:37601,127.21.81.252:42557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:32885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:59.384687 13013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:59.384923 13013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:59.389930 13018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:59.389928 13021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:59.390007 13019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:59.390038 13013 server_base.cc:1034] running on GCE node
I20260809 19:29:59.390368 13013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:59.390679 13013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:59.391845 13013 hybrid_clock.cc:648] HybridClock initialized: now 1786303799391832 us; error 31 us; skew 500 ppm
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:59.394421 13013 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:29:59.394886 13013 webserver.cc:460] Webserver started at http://127.21.81.254:46341/ using document root <none> and password file <none>
I20260809 19:29:59.395117 13013 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:59.395186 13013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:59.395349 13013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:59.396431 13013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "572da2d7668a4e95966458a9941fe287"
format_stamp: "Formatted at 2026-08-09 19:29:59 on dist-test-slave-njxd"
server_key: "7cae753160b21e0ac9de260e9b030e88"
server_key_iv: "ead79ffe94b48e9fee753be83a2d6a56"
server_key_version: "encryptionkey@0"
I20260809 19:29:59.396804 13013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "572da2d7668a4e95966458a9941fe287"
format_stamp: "Formatted at 2026-08-09 19:29:59 on dist-test-slave-njxd"
server_key: "7cae753160b21e0ac9de260e9b030e88"
server_key_iv: "ead79ffe94b48e9fee753be83a2d6a56"
server_key_version: "encryptionkey@0"
I20260809 19:29:59.398226 13013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:29:59.398983 13028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:59.399158 13013 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:29:59.399266 13013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "572da2d7668a4e95966458a9941fe287"
format_stamp: "Formatted at 2026-08-09 19:29:59 on dist-test-slave-njxd"
server_key: "7cae753160b21e0ac9de260e9b030e88"
server_key_iv: "ead79ffe94b48e9fee753be83a2d6a56"
server_key_version: "encryptionkey@0"
I20260809 19:29:59.399348 13013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:59.418435 13013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:59.420325 13013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:59.420492 13013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:59.424921 13013 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:32885
I20260809 19:29:59.424957 13081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:32885 every 8 connection(s)
I20260809 19:29:59.425344 13013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:29:59.426353 13082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:59.427920 13082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:29:59.433466 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 13013
I20260809 19:29:59.433630 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:29:59.433818 21831 external_mini_cluster.cc:1428] Setting key 56845f1b4a983420e3f40c24b12924a2
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:29:59.437927 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.428642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32885 (local address 127.0.0.1:38948)
0809 19:29:59.428894 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.428923 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:59.429374 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:59.429665 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:59.429667 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:59.429923 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:59.430078 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.430083 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.431064 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.431065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.431747 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.431749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.432215 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.432216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.432269 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.432838 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:59.432847 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:59.434431 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:59.436331 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.436339 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.436341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.436616 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.436854 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.436855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.436856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.436962 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.437434 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:59.437439 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:59.437495 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 19:29:59.437568 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 19:29:59.437761 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":104,"threads_started":1}
I20260809 19:29:59.438030 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.428675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38948 (local address 127.21.81.254:32885)
0809 19:29:59.428840 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.428848 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.429008 (+ 160us) server_negotiation.cc:407] Connection header received
0809 19:29:59.429497 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.429499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.429515 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.429651 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.430162 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.430981 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.431822 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.432123 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.432199 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.434523 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:59.434531 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:59.434531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:59.434540 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:59.436243 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.436706 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.436709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.436709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.436785 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.437055 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.437056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.437056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.437362 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:59.437451 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:59.437669 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 19:29:59.437850 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260809 19:29:59.438941 13082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:29:59.439303 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.439174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:37601 (local address 127.0.0.1:51194)
0809 19:29:59.439231 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.439276 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 19:29:59.439467 13031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.81.253:37601: connect: Connection refused (error 111)
I20260809 19:29:59.439939 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.439849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:37601 (local address 127.0.0.1:51198)
0809 19:29:59.439899 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.439923 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 19:29:59.440078 13082 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.253:37601: Network error: Client connection negotiation failed: client connection to 127.21.81.253:37601: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.21.81.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.253.keytab.
Entry for principal kudu/127.21.81.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.253.keytab.
I20260809 19:29:59.460784 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.253:37601
--webserver_interface=127.21.81.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.253.keytab
--principal=kudu/127.21.81.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:32885,127.21.81.253:37601,127.21.81.252:42557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:29:59.489378 13082 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } attempt: 1
I20260809 19:29:59.489928 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.489784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:37601 (local address 127.0.0.1:51200)
0809 19:29:59.489852 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.489898 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 19:29:59.490317 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.490231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:37601 (local address 127.0.0.1:51216)
0809 19:29:59.490273 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.490299 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 19:29:59.490425 13082 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.253:37601: Network error: Client connection negotiation failed: client connection to 127.21.81.253:37601: connect: Connection refused (error 111)
W20260809 19:29:59.543960 13090 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:29:59.544183 13090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:59.544203 13090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:59.544224 13090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:59.545406 13090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:59.545449 13090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:59.545495 13090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:59.545531 13090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:59.545554 13090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:59.547123 13090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:32885,127.21.81.253:37601,127.21.81.252:42557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.253.keytab
--principal=kudu/127.21.81.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.253:37601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.81.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:59.547338 13090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:59.547514 13090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:59.552379 13096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:59.552353 13097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:59.552599 13099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:59.552588 13090 server_base.cc:1034] running on GCE node
I20260809 19:29:59.552871 13090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:59.553180 13090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:59.554325 13090 hybrid_clock.cc:648] HybridClock initialized: now 1786303799554315 us; error 32 us; skew 500 ppm
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:59.556836 13090 init.cc:377] Logged in from keytab as kudu/127.21.81.253@KRBTEST.COM (short username kudu)
I20260809 19:29:59.557329 13090 webserver.cc:460] Webserver started at http://127.21.81.253:40613/ using document root <none> and password file <none>
I20260809 19:29:59.557654 13090 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:59.557714 13090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:59.557871 13090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:59.558553 13082 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } attempt: 2
I20260809 19:29:59.558980 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.558851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:37601 (local address 127.0.0.1:51226)
0809 19:29:59.558913 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.558950 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 19:29:59.558996 13090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/data/instance:
uuid: "a34b1d22046f43a3bff111807bfcf3fb"
format_stamp: "Formatted at 2026-08-09 19:29:59 on dist-test-slave-njxd"
server_key: "ef2749005b5afedbc8f86422c86f7b9f"
server_key_iv: "a57f27b6a5fbdf8bb8374f32c80403bb"
server_key_version: "encryptionkey@0"
I20260809 19:29:59.559396 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.559307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:37601 (local address 127.0.0.1:51236)
0809 19:29:59.559354 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.559379 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:37601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 19:29:59.559433 13090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/wal/instance:
uuid: "a34b1d22046f43a3bff111807bfcf3fb"
format_stamp: "Formatted at 2026-08-09 19:29:59 on dist-test-slave-njxd"
server_key: "ef2749005b5afedbc8f86422c86f7b9f"
server_key_iv: "a57f27b6a5fbdf8bb8374f32c80403bb"
server_key_version: "encryptionkey@0"
W20260809 19:29:59.559511 13082 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.253:37601: Network error: Client connection negotiation failed: client connection to 127.21.81.253:37601: connect: Connection refused (error 111)
I20260809 19:29:59.560788 13090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:29:59.561456 13107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:59.561676 13090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:29:59.561765 13090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/wal
uuid: "a34b1d22046f43a3bff111807bfcf3fb"
format_stamp: "Formatted at 2026-08-09 19:29:59 on dist-test-slave-njxd"
server_key: "ef2749005b5afedbc8f86422c86f7b9f"
server_key_iv: "a57f27b6a5fbdf8bb8374f32c80403bb"
server_key_version: "encryptionkey@0"
I20260809 19:29:59.561812 13090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:59.577003 13090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:59.579414 13090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:59.579531 13090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:59.583561 13090 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.253:37601
I20260809 19:29:59.583624 13162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.253:37601 every 8 connection(s)
I20260809 19:29:59.583982 13090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/data/info.pb
I20260809 19:29:59.585026 13163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:59.586196 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 13090
I20260809 19:29:59.586299 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-1/wal/instance
I20260809 19:29:59.586501 21831 external_mini_cluster.cc:1428] Setting key c50d632a7170d4f1e2d24e08e24551b5
I20260809 19:29:59.586781 13163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.253@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.81.252@KRBTEST.COM; defaulting to no policy
I20260809 19:29:59.596410 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.587529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38958 (local address 127.21.81.254:32885)
0809 19:29:59.587590 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.587592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.587826 (+ 234us) server_negotiation.cc:407] Connection header received
0809 19:29:59.588470 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.588471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.588481 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.588536 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.589392 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.589961 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.591005 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.591235 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.591313 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.593470 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:59.593472 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:59.593472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:59.593479 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:59.594990 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.595425 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.595427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.595427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.595493 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.595755 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.595756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.595756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.595874 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:59.595951 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:59.596231 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 19:29:59.596317 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:29:59.596599 13166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.587493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32885 (local address 127.0.0.1:38958)
0809 19:29:59.587691 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.587729 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:59.588348 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:59.588611 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:59.588616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:59.588981 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:59.589293 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.589299 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.590050 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.590051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.590880 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.590883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.591349 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.591350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.591469 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.591972 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:59.591989 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:59.593368 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:59.595097 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.595105 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.595107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.595346 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.595564 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.595564 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.595565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.595678 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.595942 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:59.595949 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:59.596092 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 19:29:59.596204 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 19:29:59.596360 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":102,"threads_started":1}
I20260809 19:29:59.597163 13163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.252@KRBTEST.COM" created.
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.253@KRBTEST.COM for kudu/127.21.81.253@KRBTEST.COM
I20260809 19:29:59.605424 13166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.597511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:37601 (local address 127.0.0.1:51238)
0809 19:29:59.597550 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.597561 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:59.597627 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:59.597859 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:59.597860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:59.598021 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:59.598106 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.598108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.598939 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.598939 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.599454 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.599455 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.600109 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.600110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.600173 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.600577 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:59.600584 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:59.602101 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:59.604435 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.604436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.604437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.604594 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.604826 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.604827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.604827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.604878 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.605207 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:59.605207 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:59.605240 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 19:29:59.605290 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 19:29:59.605359 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 19:29:59.605604 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.597554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51238 (local address 127.21.81.253:37601)
0809 19:29:59.597737 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.597740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.597755 (+ 15us) server_negotiation.cc:407] Connection header received
0809 19:29:59.597770 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.597771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.597781 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.597833 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.598181 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.598854 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.599546 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.600021 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.600103 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.602215 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:59.602224 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:59.602224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:59.602232 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:59.604350 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.604677 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.604681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.604681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.604755 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.604942 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.604943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.604943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.605133 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:59.605219 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:59.605335 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 19:29:59.605422 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":130,"threads_started":1}
I20260809 19:29:59.606395 13163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:29:59.606755 13166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.606631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:42557 (local address 127.0.0.1:56070)
0809 19:29:59.606682 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.606726 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 19:29:59.606918 13108 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
I20260809 19:29:59.607344 13166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.607255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:42557 (local address 127.0.0.1:56084)
0809 19:29:59.607305 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.607329 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 19:29:59.607486 13163 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.252:42557: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.252.keytab.
Entry for principal kudu/127.21.81.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.252.keytab.
I20260809 19:29:59.614167 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.252:42557
--webserver_interface=127.21.81.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.252.keytab
--principal=kudu/127.21.81.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:32885,127.21.81.253:37601,127.21.81.252:42557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:29:59.650651 13082 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } attempt: 3
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for kudu/127.21.81.253@KRBTEST.COM
I20260809 19:29:59.656739 13163 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } attempt: 1
I20260809 19:29:59.657164 13166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.657040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:42557 (local address 127.0.0.1:56088)
0809 19:29:59.657098 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.657127 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 19:29:59.657531 13166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.657400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:42557 (local address 127.0.0.1:56096)
0809 19:29:59.657458 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.657491 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 19:29:59.657672 13163 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.252:42557: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
I20260809 19:29:59.658239 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.651157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51250 (local address 127.21.81.253:37601)
0809 19:29:59.651240 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.651242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.651256 (+ 14us) server_negotiation.cc:407] Connection header received
0809 19:29:59.651286 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.651287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.651301 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.651371 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.651742 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.652255 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.652986 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.653238 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.653321 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.655488 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:59.655489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:59.655490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:59.655499 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:59.656979 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.657385 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.657386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.657386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.657474 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.657757 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.657758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.657758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.657888 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:59.657978 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:59.658116 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 19:29:59.658179 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 19:29:59.658291 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.651069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:37601 (local address 127.0.0.1:51250)
0809 19:29:59.651131 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.651149 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:59.651217 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:59.651390 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:59.651391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:59.651564 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:59.651636 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.651638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.652360 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.652361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.652897 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.652899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.653333 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.653334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.653417 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.653932 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:59.653937 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:59.655389 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:59.657074 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.657076 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.657076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.657304 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.657557 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.657558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.657558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.657653 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.657960 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:59.657961 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:59.658017 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 19:29:59.658089 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 19:29:59.658197 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 19:29:59.658808 13082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:29:59.659144 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.659041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:42557 (local address 127.0.0.1:56110)
0809 19:29:59.659090 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.659122 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 19:29:59.659482 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.659384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:42557 (local address 127.0.0.1:56126)
0809 19:29:59.659436 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.659463 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 19:29:59.659571 13082 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.252:42557: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
I20260809 19:29:59.690733 13082 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } attempt: 1
I20260809 19:29:59.691294 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.691137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:42557 (local address 127.0.0.1:56132)
0809 19:29:59.691207 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.691254 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 19:29:59.691697 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.691592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:42557 (local address 127.0.0.1:56142)
0809 19:29:59.691640 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.691672 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 19:29:59.691812 13082 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.252:42557: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
W20260809 19:29:59.700461 13172 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:29:59.700663 13172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:29:59.700681 13172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:29:59.700701 13172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:29:59.701898 13172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:29:59.701939 13172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:29:59.701953 13172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:29:59.701967 13172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:29:59.702011 13172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:29:59.703634 13172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:32885,127.21.81.253:37601,127.21.81.252:42557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.252.keytab
--principal=kudu/127.21.81.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.252:42557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.81.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:29:59.703902 13172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:29:59.704185 13172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:29:59.708994 13180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:59.709038 13178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:29:59.708995 13177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:29:59.709066 13172 server_base.cc:1034] running on GCE node
I20260809 19:29:59.709445 13172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:29:59.709767 13172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:29:59.710930 13172 hybrid_clock.cc:648] HybridClock initialized: now 1786303799710925 us; error 34 us; skew 500 ppm
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:29:59.713486 13172 init.cc:377] Logged in from keytab as kudu/127.21.81.252@KRBTEST.COM (short username kudu)
I20260809 19:29:59.714000 13172 webserver.cc:460] Webserver started at http://127.21.81.252:40595/ using document root <none> and password file <none>
I20260809 19:29:59.714241 13172 fs_manager.cc:359] Metadata directory not provided
I20260809 19:29:59.714310 13172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:29:59.714464 13172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:29:59.715554 13172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/data/instance:
uuid: "b13e5def487d4ed8a9db088b1126f84c"
format_stamp: "Formatted at 2026-08-09 19:29:59 on dist-test-slave-njxd"
server_key: "0e3c9907a58688af8c673bb768208954"
server_key_iv: "4a8ba77e9bb1f30f8bd9d0b5a2f54a32"
server_key_version: "encryptionkey@0"
I20260809 19:29:59.715926 13172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/wal/instance:
uuid: "b13e5def487d4ed8a9db088b1126f84c"
format_stamp: "Formatted at 2026-08-09 19:29:59 on dist-test-slave-njxd"
server_key: "0e3c9907a58688af8c673bb768208954"
server_key_iv: "4a8ba77e9bb1f30f8bd9d0b5a2f54a32"
server_key_version: "encryptionkey@0"
I20260809 19:29:59.717308 13172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:29:59.718096 13187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:59.718261 13172 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:29:59.718364 13172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/wal
uuid: "b13e5def487d4ed8a9db088b1126f84c"
format_stamp: "Formatted at 2026-08-09 19:29:59 on dist-test-slave-njxd"
server_key: "0e3c9907a58688af8c673bb768208954"
server_key_iv: "4a8ba77e9bb1f30f8bd9d0b5a2f54a32"
server_key_version: "encryptionkey@0"
I20260809 19:29:59.718451 13172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:29:59.725823 13163 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } attempt: 2
I20260809 19:29:59.726262 13166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.726110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:42557 (local address 127.0.0.1:56156)
0809 19:29:59.726176 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.726225 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 19:29:59.726624 13166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.726538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:42557 (local address 127.0.0.1:56164)
0809 19:29:59.726581 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.726608 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 19:29:59.726738 13163 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.252:42557: Network error: Client connection negotiation failed: client connection to 127.21.81.252:42557: connect: Connection refused (error 111)
I20260809 19:29:59.739892 13172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:29:59.741915 13172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:29:59.742079 13172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:29:59.746603 13172 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.252:42557
I20260809 19:29:59.746657 13248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.252:42557 every 8 connection(s)
I20260809 19:29:59.747038 13172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/data/info.pb
I20260809 19:29:59.748049 13249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:29:59.749411 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 13172
I20260809 19:29:59.749536 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303525464116-21831-0/minicluster-data/master-2/wal/instance
I20260809 19:29:59.749786 21831 external_mini_cluster.cc:1428] Setting key 2416b32d8faca285a64d119d420aa37e
I20260809 19:29:59.750102 13249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.252@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.252@KRBTEST.COM
I20260809 19:29:59.759415 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.750985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38970 (local address 127.21.81.254:32885)
0809 19:29:59.751035 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.751037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.751161 (+ 124us) server_negotiation.cc:407] Connection header received
0809 19:29:59.751305 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.751306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.751318 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.751373 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.751938 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.752429 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.753334 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.753530 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.753623 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.756043 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:59.756045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:59.756045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:59.756053 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:59.757811 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.758296 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.758298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.758298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.758365 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.758667 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.758668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.758668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.758819 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:59.758894 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:59.759227 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 19:29:59.759317 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:29:59.759599 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.750894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:32885 (local address 127.0.0.1:38970)
0809 19:29:59.751038 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.751087 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:59.751223 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:59.751390 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:59.751393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:59.751675 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:59.751838 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.751842 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.752517 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.752518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.753223 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.753225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.753664 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.753665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.753783 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.754322 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:59.754333 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:59.755949 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:59.757902 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.757909 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.757910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.758223 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.758436 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.758437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.758437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.758597 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.758909 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:59.758917 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:59.759096 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 19:29:59.759179 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:29:59.759361 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260809 19:29:59.759867 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.750365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56170 (local address 127.21.81.252:42557)
0809 19:29:59.750579 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.750587 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.750614 (+ 27us) server_negotiation.cc:407] Connection header received
0809 19:29:59.751203 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.751210 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.751232 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.751345 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.751988 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.752779 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.753521 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.753881 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.753964 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.756550 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:59.756557 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:59.756557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:59.756563 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:59.758418 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.758802 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.758805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.758805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.758862 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.759061 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.759061 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.759061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.759234 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:59.759410 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:59.759480 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 19:29:59.759569 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":100,"threads_started":1}
I20260809 19:29:59.760501 13249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.252@KRBTEST.COM for kudu/127.21.81.253@KRBTEST.COM
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:29:59.766973 13082 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } attempt: 2
I20260809 19:29:59.768303 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.760841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51260 (local address 127.21.81.253:37601)
0809 19:29:59.760886 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.760888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.760897 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:29:59.760923 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.760924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.760934 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.760989 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.761270 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.761909 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.762629 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.762938 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.762988 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.765412 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:59.765414 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:59.765414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:59.765422 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:59.767038 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.767441 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.767442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.767442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.767502 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.767697 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.767698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.767698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.767846 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:59.767924 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:59.768123 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 19:29:59.768207 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:29:59.768611 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.760768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:37601 (local address 127.0.0.1:51260)
0809 19:29:59.760810 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.760819 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:59.760856 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:59.761002 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:59.761003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:59.761154 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:59.761190 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.761191 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.761977 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.761978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.762553 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.762554 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.763002 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.763003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.763067 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.763483 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:59.763487 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:59.765322 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:59.767134 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.767135 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.767135 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.767356 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.767579 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.767579 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.767579 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.767635 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.767956 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:59.767957 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:59.768000 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 19:29:59.768428 (+ 428us) client_negotiation.cc:240] Negotiation successful
0809 19:29:59.768521 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 19:29:59.768636 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.760802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38972 (local address 127.21.81.254:32885)
0809 19:29:59.760838 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.760840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.760892 (+ 52us) server_negotiation.cc:407] Connection header received
0809 19:29:59.760980 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.760981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.760990 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.761044 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.761627 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.762138 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.762857 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.763073 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.763143 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.765506 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:59.765507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:59.765507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:59.765514 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:59.767186 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.767643 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.767644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.767644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.767705 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.767905 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.767906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.767907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.768031 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:59.768191 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:59.768253 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 19:29:59.768323 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:29:59.768999 13249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for kudu/127.21.81.252@KRBTEST.COM
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.252@KRBTEST.COM for kudu/127.21.81.252@KRBTEST.COM
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.253@KRBTEST.COM
I20260809 19:29:59.774878 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.767299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56184 (local address 127.21.81.252:42557)
0809 19:29:59.767351 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.767353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.767360 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:29:59.767381 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.767382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.767393 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.767450 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.767739 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.768392 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.769351 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.769688 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.770007 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.771678 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:59.771679 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:59.771679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:59.771685 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:59.773779 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.774128 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.774130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.774130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.774185 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.774362 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.774363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.774363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.774469 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:59.774541 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:59.774726 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 19:29:59.774793 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:29:59.774878 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.767240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:42557 (local address 127.0.0.1:56184)
0809 19:29:59.767293 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.767304 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:59.767340 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:59.767453 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:59.767453 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:59.767615 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:59.767659 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.767661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.768493 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.768494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.769249 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.769251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.769792 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.769794 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.769851 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.770287 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:59.770292 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:59.771586 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:59.773862 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.773863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.773864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.774039 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.774238 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.774239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.774239 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.774294 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.774548 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:59.774549 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:59.774606 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 19:29:59.774681 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 19:29:59.774782 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 19:29:59.776455 13082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287: Bootstrap starting.
I20260809 19:29:59.777617 13082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:59.777730 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.769402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:42557 (local address 127.0.0.1:56194)
0809 19:29:59.769437 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.769446 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:59.769474 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:59.769792 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:59.769794 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:59.769950 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:59.769985 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.769987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.770910 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.770911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.771665 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.771668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.772137 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.772138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.772209 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.772525 (+ 316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:59.772529 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:59.773836 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:59.776642 (+ 2806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.776643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.776643 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.776858 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.777092 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.777092 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.777093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.777142 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.777442 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:59.777443 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:59.777480 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 19:29:59.777527 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:29:59.777644 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 19:29:59.777779 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.769238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51276 (local address 127.21.81.253:37601)
0809 19:29:59.769338 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.769340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.769350 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:29:59.769393 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.769394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.769404 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.769455 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.770155 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.770843 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.771578 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.771851 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.771907 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.774745 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:59.774747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:59.774748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:59.774755 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:59.776597 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.776931 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.776932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.776932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.776986 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.777219 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.777220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.777220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.777331 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:59.777516 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:59.777553 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 19:29:59.777664 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 19:29:59.777973 13082 log.cc:824] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:59.778126 13255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.769476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56194 (local address 127.21.81.252:42557)
0809 19:29:59.769664 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.769666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.769672 (+ 6us) server_negotiation.cc:407] Connection header received
0809 19:29:59.769689 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.769690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.769699 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.769769 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.770069 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.770824 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.771758 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.772057 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.772131 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.773940 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:59.773941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:59.773942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:59.773948 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:59.776533 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.776953 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.776953 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.776954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.777019 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.777221 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.777222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.777222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.777366 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:59.777910 (+ 544us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:59.777951 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 19:29:59.778023 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":124,"threads_started":1}
I20260809 19:29:59.778867 13082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287: No bootstrap required, opened a new log
I20260809 19:29:59.779161 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:29:59.779516 13249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c: Bootstrap starting.
I20260809 19:29:59.780108 13249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:59.780138 13056 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "572da2d7668a4e95966458a9941fe287"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38972
I20260809 19:29:59.780436 13249 log.cc:824] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:59.781031 13082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } }
I20260809 19:29:59.781168 13082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 572da2d7668a4e95966458a9941fe287, State: Initialized, Role: FOLLOWER
I20260809 19:29:59.781208 13249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c: No bootstrap required, opened a new log
I20260809 19:29:59.781241 13082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } }
I20260809 19:29:59.781527 13259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } } }
I20260809 19:29:59.781626 13056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 19:29:59.781759 13259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:29:59.781816 13056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:59.782575 13056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } }
I20260809 19:29:59.782843 13056 leader_election.cc:288] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [CANDIDATE]: Term 1 election: Requested vote from peers a34b1d22046f43a3bff111807bfcf3fb (127.21.81.253:37601), b13e5def487d4ed8a9db088b1126f84c (127.21.81.252:42557)
I20260809 19:29:59.781626 13082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:59.783106 13249 raft_consensus.cc:348] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } }
I20260809 19:29:59.783258 13249 raft_consensus.cc:729] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13e5def487d4ed8a9db088b1126f84c, State: Initialized, Role: FOLLOWER
I20260809 19:29:59.783366 13249 consensus_queue.cc:260] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } }
I20260809 19:29:59.783733 13249 sys_catalog.cc:564] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:59.783795 13218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "572da2d7668a4e95966458a9941fe287" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b13e5def487d4ed8a9db088b1126f84c"
I20260809 19:29:59.783917 13218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:59.783915 13265 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } } }
I20260809 19:29:59.784014 13265 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:29:59.784816 13218 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 572da2d7668a4e95966458a9941fe287 in term 1.
I20260809 19:29:59.785105 13029 leader_election.cc:302] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 572da2d7668a4e95966458a9941fe287, b13e5def487d4ed8a9db088b1126f84c; no voters:
I20260809 19:29:59.786288 13259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:29:59.786444 13259 raft_consensus.cc:686] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [term 1 LEADER]: Becoming Leader. State: Replica: 572da2d7668a4e95966458a9941fe287, State: Running, Role: LEADER
I20260809 19:29:59.786547 13259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } }
I20260809 19:29:59.786996 13294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 572da2d7668a4e95966458a9941fe287. Latest consensus state: current_term: 1 leader_uuid: "572da2d7668a4e95966458a9941fe287" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } } }
I20260809 19:29:59.787240 13294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [sys.catalog]: This master's current role is: LEADER
W20260809 19:29:59.785192 13282 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:29:59.787774 13282 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 19:29:59.788499 13291 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:29:59.788556 13291 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:29:59.788556 13013 master_runner.cc:186] Error getting master registration for 127.21.81.253:37601: OK, instance_id { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" instance_seqno: 1786303799582443 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:29:59.791134 13013 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:29:59.789677 13172 master_runner.cc:186] Error getting master registration for 127.21.81.253:37601: OK, instance_id { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" instance_seqno: 1786303799582443 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:29:59.791980 13172 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:29:59.796229 13255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.785015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56204 (local address 127.21.81.252:42557)
0809 19:29:59.785327 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.785330 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.785753 (+ 423us) server_negotiation.cc:407] Connection header received
0809 19:29:59.785861 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.785862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.785874 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.785903 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.786698 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.787399 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.788454 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.788783 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.788830 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.793156 (+ 4326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:59.793157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:59.793158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:59.793165 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:59.794763 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.795243 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.795244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.795245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.795302 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.795562 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.795563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.795563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.795718 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:59.795809 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:59.796037 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 19:29:59.796121 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272}
I20260809 19:29:59.796656 13084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.785594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38982 (local address 127.21.81.254:32885)
0809 19:29:59.785634 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.785637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.785928 (+ 291us) server_negotiation.cc:407] Connection header received
0809 19:29:59.786052 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.786053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.786063 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.786121 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.786819 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.787507 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.791501 (+ 3994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.791764 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.791808 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.793328 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:59.793330 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:59.793330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:59.793338 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:59.794873 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.795780 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.795781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.795781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.795836 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.796034 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.796035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.796036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.796155 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:59.796264 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:59.796488 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 19:29:59.796570 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:29:59.797137 13296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:29:59.797396 13296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:29:59.799659 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.785790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.21.81.253:37601)
0809 19:29:59.785828 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.785831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.788281 (+ 2450us) server_negotiation.cc:407] Connection header received
0809 19:29:59.788303 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.788304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.788314 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.788367 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.790083 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.790786 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.792473 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.792763 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.792809 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.795852 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:59.795853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:59.795854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:59.795861 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:59.797913 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.798529 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.798531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.798532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.798596 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.798897 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.798898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.798898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.799048 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:59.799117 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:59.799478 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 19:29:59.799566 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:29:59.810639 13218 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c [term 1 FOLLOWER]: Refusing update from remote peer 572da2d7668a4e95966458a9941fe287: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:29:59.811008 13294 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:29:59.812527 13265 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 572da2d7668a4e95966458a9941fe287. Latest consensus state: current_term: 1 leader_uuid: "572da2d7668a4e95966458a9941fe287" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } } }
I20260809 19:29:59.812626 13265 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:29:59.814529 13259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "572da2d7668a4e95966458a9941fe287" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } } }
I20260809 19:29:59.814695 13259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [sys.catalog]: This master's current role is: LEADER
I20260809 19:29:59.814710 13265 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "572da2d7668a4e95966458a9941fe287" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } } }
I20260809 19:29:59.814788 13265 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:29:59.814947 13298 mvcc.cc:205] Tried to move back new op lower bound from 7316700364021354496 to 7316700363926396928. Current Snapshot: MvccSnapshot[applied={T|T < 7316700364021354496}]
I20260809 19:29:59.815048 13299 mvcc.cc:205] Tried to move back new op lower bound from 7316700364021354496 to 7316700363926396928. Current Snapshot: MvccSnapshot[applied={T|T < 7316700364021354496}]
I20260809 19:29:59.815869 13296 catalog_manager.cc:1347] Generated new cluster ID: 1fa34dea4fe04e4baba1a7c6755ca1cb
I20260809 19:29:59.815934 13296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:29:59.817888 13163 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } attempt: 3
Aug 09 19:29:59 dist-test-slave-njxd krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303799, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.253@KRBTEST.COM for kudu/127.21.81.252@KRBTEST.COM
I20260809 19:29:59.826299 13296 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:29:59.826961 13166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.818154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:42557 (local address 127.0.0.1:56210)
0809 19:29:59.818206 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 19:29:59.818218 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:29:59.818258 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:29:59.818394 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:29:59.818395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:29:59.818589 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:29:59.818662 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.818664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.819615 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.819616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.820475 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:29:59.820478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.821119 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.821120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:29:59.821194 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.821910 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:29:59.821916 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:29:59.823513 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:29:59.825630 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.825632 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.825632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.825864 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.826134 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:29:59.826135 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:29:59.826135 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:29:59.826229 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:29:59.826628 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:29:59.826629 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:29:59.826688 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 19:29:59.826752 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 19:29:59.826854 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 19:29:59.827006 13296 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:29:59.827524 13255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:29:59.818196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56210 (local address 127.21.81.252:42557)
0809 19:29:59.818252 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.818254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.818267 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:29:59.818297 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.818298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.818307 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.818381 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.818753 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.819505 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.820609 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:29:59.820981 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:29:59.821043 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:29:59.823623 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:29:59.823624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:29:59.823625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:29:59.823632 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:29:59.825487 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.825963 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.825964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.825965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.826039 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:29:59.826313 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:29:59.826314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:29:59.826314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:29:59.826464 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:29:59.826522 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:29:59.827353 (+ 831us) server_negotiation.cc:299] Negotiation successful
0809 19:29:59.827434 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 19:29:59.829402 13163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb: Bootstrap starting.
I20260809 19:29:59.830291 13163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb: Neither blocks nor log segments found. Creating new log.
I20260809 19:29:59.830617 13163 log.cc:824] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb: Log is configured to *not* fsync() on all Append() calls
I20260809 19:29:59.831925 13296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287: Generated new TSK 0
I20260809 19:29:59.832086 13296 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:29:59.841244 13163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb: No bootstrap required, opened a new log
I20260809 19:29:59.843448 13163 raft_consensus.cc:348] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } }
I20260809 19:29:59.843603 13163 raft_consensus.cc:729] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34b1d22046f43a3bff111807bfcf3fb, State: Initialized, Role: FOLLOWER
I20260809 19:29:59.843690 13163 consensus_queue.cc:260] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } }
I20260809 19:29:59.844031 13163 sys_catalog.cc:564] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:29:59.844301 13135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "572da2d7668a4e95966458a9941fe287" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a34b1d22046f43a3bff111807bfcf3fb"
I20260809 19:29:59.844565 13134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:29:59.845538 13134 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb [term 1 FOLLOWER]: Refusing update from remote peer 572da2d7668a4e95966458a9941fe287: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:29:59.845909 13259 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:29:59.846565 13135 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 572da2d7668a4e95966458a9941fe287 in term 1.
I20260809 19:29:59.847213 13307 sys_catalog.cc:455] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } } }
I20260809 19:29:59.847363 13307 sys_catalog.cc:458] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:29:59.848038 13307 sys_catalog.cc:455] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 572da2d7668a4e95966458a9941fe287. Latest consensus state: current_term: 1 leader_uuid: "572da2d7668a4e95966458a9941fe287" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } } }
I20260809 19:29:59.848129 13307 sys_catalog.cc:458] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:29:59.848163 13307 sys_catalog.cc:455] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "572da2d7668a4e95966458a9941fe287" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } } }
I20260809 19:29:59.848220 13307 sys_catalog.cc:458] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:29:59.848524 13322 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:29:59.848631 13322 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:29:59.850178 13320 mvcc.cc:205] Tried to move back new op lower bound from 7316700364107538432 to 7316700363926396928. Current Snapshot: MvccSnapshot[applied={T|T < 7316700364021354496}]
W20260809 19:29:59.870579 13255 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:29:59.868871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56222 (local address 127.21.81.252:42557)
0809 19:29:59.868919 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.868921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.869906 (+ 985us) server_negotiation.cc:407] Connection header received
0809 19:29:59.870021 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.870022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.870033 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.870065 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.870507 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56222: BlockingRecv error: recv got EOF from 127.0.0.1:56222 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 19:29:59.871325 13084 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:29:59.869167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38986 (local address 127.21.81.254:32885)
0809 19:29:59.869209 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.869211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.870824 (+ 1613us) server_negotiation.cc:407] Connection header received
0809 19:29:59.870919 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.870920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.870931 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.870960 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.871263 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38986: BlockingRecv error: recv got EOF from 127.0.0.1:38986 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 19:29:59.872044 13169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:29:59.869697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51302 (local address 127.21.81.253:37601)
0809 19:29:59.869743 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.869745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.871544 (+ 1799us) server_negotiation.cc:407] Connection header received
0809 19:29:59.871641 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.871642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.871652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.871682 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.871985 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51302: BlockingRecv error: recv got EOF from 127.0.0.1:51302 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260809 19:29:59.874392 13169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:29:59.873630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51314 (local address 127.21.81.253:37601)
0809 19:29:59.873664 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.873666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.873887 (+ 221us) server_negotiation.cc:407] Connection header received
0809 19:29:59.873990 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.873991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.874011 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.874041 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.874349 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51314: BlockingRecv error: recv got EOF from 127.0.0.1:51314 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 19:29:59.877369 13169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:29:59.876622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51322 (local address 127.21.81.253:37601)
0809 19:29:59.876656 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.876658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.876911 (+ 253us) server_negotiation.cc:407] Connection header received
0809 19:29:59.877005 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.877006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.877016 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.877044 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.877326 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51322: BlockingRecv error: recv got EOF from 127.0.0.1:51322 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 19:29:59.881124 13169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:29:59.880426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51328 (local address 127.21.81.253:37601)
0809 19:29:59.880459 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.880461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.880660 (+ 199us) server_negotiation.cc:407] Connection header received
0809 19:29:59.880762 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.880763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.880772 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.880800 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.881086 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51328: BlockingRecv error: recv got EOF from 127.0.0.1:51328 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 19:29:59.887059 13169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:29:59.886251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51334 (local address 127.21.81.253:37601)
0809 19:29:59.886286 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.886288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.886541 (+ 253us) server_negotiation.cc:407] Connection header received
0809 19:29:59.886643 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.886644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.886655 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.886685 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.887015 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51334: BlockingRecv error: recv got EOF from 127.0.0.1:51334 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 19:29:59.897200 13169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:29:59.896292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.21.81.253:37601)
0809 19:29:59.896331 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.896333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.896606 (+ 273us) server_negotiation.cc:407] Connection header received
0809 19:29:59.896729 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.896730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.896742 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.896773 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.897136 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51350: BlockingRecv error: recv got EOF from 127.0.0.1:51350 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 19:29:59.915482 13169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:29:59.914513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51360 (local address 127.21.81.253:37601)
0809 19:29:59.914555 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.914572 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.914916 (+ 344us) server_negotiation.cc:407] Connection header received
0809 19:29:59.915023 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.915025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.915037 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.915068 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.915428 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51360: BlockingRecv error: recv got EOF from 127.0.0.1:51360 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 19:29:59.949849 13169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:29:59.948868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51372 (local address 127.21.81.253:37601)
0809 19:29:59.948913 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:29:59.948915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:29:59.949222 (+ 307us) server_negotiation.cc:407] Connection header received
0809 19:29:59.949326 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:29:59.949327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:29:59.949339 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:29:59.949370 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:29:59.949781 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51372: BlockingRecv error: recv got EOF from 127.0.0.1:51372 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 19:30:00.016084 13169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:00.015387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51378 (local address 127.21.81.253:37601)
0809 19:30:00.015442 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:00.015444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:00.015554 (+ 110us) server_negotiation.cc:407] Connection header received
0809 19:30:00.015587 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:00.015588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:00.015600 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:00.015681 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:00.016018 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51378: BlockingRecv error: recv got EOF from 127.0.0.1:51378 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260809 19:30:00.146437 13169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:00.145708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51386 (local address 127.21.81.253:37601)
0809 19:30:00.145791 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:00.145793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:00.145806 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:30:00.145840 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:00.145841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:00.145902 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:00.145990 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:00.146385 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51386: BlockingRecv error: recv got EOF from 127.0.0.1:51386 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 19:30:00.404409 13169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:00.403682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51390 (local address 127.21.81.253:37601)
0809 19:30:00.403760 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:00.403762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:00.403772 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:30:00.403877 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:00.403878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:00.403890 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:00.403965 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:00.404337 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51390: BlockingRecv error: recv got EOF from 127.0.0.1:51390 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 19:30:00.429852 13391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "572da2d7668a4e95966458a9941fe287" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "572da2d7668a4e95966458a9941fe287" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 32885 } } peers { permanent_uuid: "a34b1d22046f43a3bff111807bfcf3fb" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 37601 } } peers { permanent_uuid: "b13e5def487d4ed8a9db088b1126f84c" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 42557 } } }
I20260809 19:30:00.430023 13391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 572da2d7668a4e95966458a9941fe287 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:00.788964 13291 catalog_manager.cc:1259] Loaded cluster ID: 1fa34dea4fe04e4baba1a7c6755ca1cb
I20260809 19:30:00.789032 13291 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c: loading cluster ID for follower catalog manager: success
I20260809 19:30:00.789808 13291 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c: acquiring CA information for follower catalog manager: success
I20260809 19:30:00.790057 13291 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b13e5def487d4ed8a9db088b1126f84c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:30:00.849134 13322 catalog_manager.cc:1259] Loaded cluster ID: 1fa34dea4fe04e4baba1a7c6755ca1cb
I20260809 19:30:00.849244 13322 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb: loading cluster ID for follower catalog manager: success
I20260809 19:30:00.850139 13322 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb: acquiring CA information for follower catalog manager: success
I20260809 19:30:00.850471 13322 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a34b1d22046f43a3bff111807bfcf3fb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:30:00.920104 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:00.917640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.21.81.253:37601)
0809 19:30:00.917751 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:00.917754 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:00.917794 (+ 40us) server_negotiation.cc:407] Connection header received
0809 19:30:00.917872 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:00.917873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:00.917927 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:00.918002 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:00.918137 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:00.918642 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:00.919382 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:00.919573 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:00.919622 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:00.919764 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:00.919873 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:00.919929 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:00.919964 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 19:30:00.919981 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":69,"threads_started":1}
I20260809 19:30:00.924429 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:00.922203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56228 (local address 127.21.81.252:42557)
0809 19:30:00.922379 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:00.922381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:00.922388 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:00.922429 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:00.922430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:00.922465 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:00.922536 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:00.922641 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:00.923116 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:00.923787 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:00.923979 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:00.924025 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:00.924144 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:00.924244 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:00.924297 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:00.924331 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 19:30:00.924346 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":104,"threads_started":1}
I20260809 19:30:00.928925 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:00.926290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56242 (local address 127.21.81.252:42557)
0809 19:30:00.926332 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:00.926334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:00.926432 (+ 98us) server_negotiation.cc:407] Connection header received
0809 19:30:00.926496 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:00.926497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:00.926508 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:00.926551 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:00.926644 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:00.927336 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:00.928236 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:00.928443 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:00.928517 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:00.928662 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:00.928692 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:00.928718 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:00.928809 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 19:30:00.928835 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:30:00.929381 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:00.926360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51412 (local address 127.21.81.253:37601)
0809 19:30:00.926402 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:00.926404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:00.926791 (+ 387us) server_negotiation.cc:407] Connection header received
0809 19:30:00.926828 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:00.926829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:00.926841 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:00.926898 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:00.927019 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:00.927514 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:00.928580 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:00.928891 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:00.928956 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:00.929092 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:00.929128 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:00.929188 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:00.929265 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 19:30:00.929288 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:30:00.929875 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:00.926239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38992 (local address 127.21.81.254:32885)
0809 19:30:00.926463 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:00.926465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:00.926473 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:00.926523 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:00.926524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:00.926559 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:00.926643 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:00.926957 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:00.927733 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:00.928741 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:00.929135 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:00.929206 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:00.929442 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:00.929553 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:00.929636 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:00.929745 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 19:30:00.929778 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":51,"threads_started":1}
I20260809 19:30:00.931502 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 13013
I20260809 19:30:00.939822 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 13090
I20260809 19:30:00.946977 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 13172
2026-08-09T19:30:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2837 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:30:01 dist-test-slave-njxd krb5kdc[13418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:30:01 dist-test-slave-njxd krb5kdc[13418](info): set up 2 sockets
Aug 09 19:30:01 dist-test-slave-njxd krb5kdc[13418](info): commencing operation
krb5kdc: starting...
W20260809 19:30:02.017258 21831 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/test-user.keytab.
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:30:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:30:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.81.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
Entry for principal kudu/127.21.81.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab.
I20260809 19:30:02.136119 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:45875
--webserver_interface=127.21.81.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:46705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:45875,127.21.81.253:36645,127.21.81.252:36663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:30:02.218514 13433 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:30:02.218792 13433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:02.218823 13433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:02.218899 13433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:02.220121 13433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:02.220168 13433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:02.220184 13433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:02.220197 13433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:02.220248 13433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:02.221843 13433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:45875,127.21.81.253:36645,127.21.81.252:36663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.254.keytab
--principal=kudu/127.21.81.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.254:45875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.81.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:02.222101 13433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:02.222338 13433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:02.227198 13439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:02.227214 13442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:02.227277 13433 server_base.cc:1034] running on GCE node
W20260809 19:30:02.227214 13440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:02.227613 13433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:02.227909 13433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:02.229053 13433 hybrid_clock.cc:648] HybridClock initialized: now 1786303802229039 us; error 29 us; skew 500 ppm
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:02.235868 13433 init.cc:377] Logged in from keytab as kudu/127.21.81.254@KRBTEST.COM (short username kudu)
I20260809 19:30:02.236326 13433 webserver.cc:460] Webserver started at http://127.21.81.254:38523/ using document root <none> and password file <none>
I20260809 19:30:02.236552 13433 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:02.236596 13433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:02.236745 13433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:02.237898 13433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/data/instance:
uuid: "6ae6447c32684494be9023f73db5bc08"
format_stamp: "Formatted at 2026-08-09 19:30:02 on dist-test-slave-njxd"
server_key: "64abd812e433b890ab2784494082f87b"
server_key_iv: "9c7bf49ec18cfab1301839890f4ba988"
server_key_version: "encryptionkey@0"
I20260809 19:30:02.238270 13433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/wal/instance:
uuid: "6ae6447c32684494be9023f73db5bc08"
format_stamp: "Formatted at 2026-08-09 19:30:02 on dist-test-slave-njxd"
server_key: "64abd812e433b890ab2784494082f87b"
server_key_iv: "9c7bf49ec18cfab1301839890f4ba988"
server_key_version: "encryptionkey@0"
I20260809 19:30:02.239614 13433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:02.240339 13451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:02.240516 13433 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:02.240631 13433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/wal
uuid: "6ae6447c32684494be9023f73db5bc08"
format_stamp: "Formatted at 2026-08-09 19:30:02 on dist-test-slave-njxd"
server_key: "64abd812e433b890ab2784494082f87b"
server_key_iv: "9c7bf49ec18cfab1301839890f4ba988"
server_key_version: "encryptionkey@0"
I20260809 19:30:02.240733 13433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:02.262601 13433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:02.265014 13433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:02.265195 13433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:02.269560 13433 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.254:45875
I20260809 19:30:02.269662 13507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.254:45875 every 8 connection(s)
I20260809 19:30:02.270049 13433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/data/info.pb
I20260809 19:30:02.271093 13508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:02.271107 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 13433
I20260809 19:30:02.271209 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-0/wal/instance
I20260809 19:30:02.271404 21831 external_mini_cluster.cc:1428] Setting key 4e81f238ce1992ba810dae636aa8d251
I20260809 19:30:02.272779 13508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.81.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.253@KRBTEST.COM" created.
I20260809 19:30:02.284116 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.273804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36196 (local address 127.21.81.254:45875)
0809 19:30:02.273922 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.273927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.273981 (+ 54us) server_negotiation.cc:407] Connection header received
0809 19:30:02.274808 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.274810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.274827 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.274894 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.275700 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.276501 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.277941 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.278244 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.278325 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.280794 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:02.280803 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:02.280803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:02.280811 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:02.282547 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.282986 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.282988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.282988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.283066 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.283341 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.283342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.283342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.283549 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:02.283617 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:02.283848 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 19:30:02.283954 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":88,"thread_start_us":62,"threads_started":1}
I20260809 19:30:02.284240 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.273568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:45875 (local address 127.0.0.1:36196)
0809 19:30:02.273833 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.273889 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:02.274645 (+ 756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:02.274937 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:02.274939 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:02.275353 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:02.275597 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.275605 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.276601 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.276602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.277857 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.277860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.278355 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.278356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.278436 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.279000 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:02.279011 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:02.280704 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:02.282623 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.282637 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.282638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.282843 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.283149 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.283149 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.283150 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.283272 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.283621 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:02.283635 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:02.283717 (+ 82us) client_negotiation.cc:769] Sending connection context
0809 19:30:02.283825 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 19:30:02.284050 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":63,"thread_start_us":100,"threads_started":1}
I20260809 19:30:02.285027 13508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:30:02.285440 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.285313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:36645 (local address 127.0.0.1:36430)
0809 19:30:02.285369 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.285419 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:36645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 19:30:02.285640 13454 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.81.253:36645: connect: Connection refused (error 111)
I20260809 19:30:02.286049 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.285963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:36645 (local address 127.0.0.1:36434)
0809 19:30:02.286006 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.286032 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:36645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 19:30:02.286178 13508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.253:36645: Network error: Client connection negotiation failed: client connection to 127.21.81.253:36645: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.253.keytab.
Entry for principal kudu/127.21.81.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.253.keytab.
I20260809 19:30:02.297807 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.253:36645
--webserver_interface=127.21.81.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.253.keytab
--principal=kudu/127.21.81.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:46705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:45875,127.21.81.253:36645,127.21.81.252:36663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:30:02.335453 13508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } attempt: 1
I20260809 19:30:02.336059 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.335893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:36645 (local address 127.0.0.1:36436)
0809 19:30:02.335967 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.336019 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:36645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 19:30:02.336463 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.336366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:36645 (local address 127.0.0.1:36448)
0809 19:30:02.336412 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.336442 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:36645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 19:30:02.336591 13508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.253:36645: Network error: Client connection negotiation failed: client connection to 127.21.81.253:36645: connect: Connection refused (error 111)
W20260809 19:30:02.380430 13516 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:30:02.380648 13516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:02.380664 13516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:02.380685 13516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:02.381963 13516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:02.382005 13516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:02.382020 13516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:02.382071 13516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:02.382087 13516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:02.383627 13516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:45875,127.21.81.253:36645,127.21.81.252:36663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.253.keytab
--principal=kudu/127.21.81.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.253:36645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.81.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:02.383904 13516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:02.384163 13516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:02.389205 13521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:02.389222 13522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:02.389252 13526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:02.389254 13516 server_base.cc:1034] running on GCE node
I20260809 19:30:02.389720 13516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:02.390025 13516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:02.391192 13516 hybrid_clock.cc:648] HybridClock initialized: now 1786303802391181 us; error 41 us; skew 500 ppm
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:02.393919 13516 init.cc:377] Logged in from keytab as kudu/127.21.81.253@KRBTEST.COM (short username kudu)
I20260809 19:30:02.394418 13516 webserver.cc:460] Webserver started at http://127.21.81.253:39675/ using document root <none> and password file <none>
I20260809 19:30:02.394659 13516 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:02.394731 13516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:02.394882 13516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:02.395956 13516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/data/instance:
uuid: "a9b433ca1896463f8edbcca120f2d43a"
format_stamp: "Formatted at 2026-08-09 19:30:02 on dist-test-slave-njxd"
server_key: "ef0e097e27d62f499b4523118ef008e0"
server_key_iv: "3a643c455be04e66254fa91f4cfee180"
server_key_version: "encryptionkey@0"
I20260809 19:30:02.396324 13516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/wal/instance:
uuid: "a9b433ca1896463f8edbcca120f2d43a"
format_stamp: "Formatted at 2026-08-09 19:30:02 on dist-test-slave-njxd"
server_key: "ef0e097e27d62f499b4523118ef008e0"
server_key_iv: "3a643c455be04e66254fa91f4cfee180"
server_key_version: "encryptionkey@0"
I20260809 19:30:02.397704 13516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:30:02.398501 13533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:02.398681 13516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:02.398795 13516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/wal
uuid: "a9b433ca1896463f8edbcca120f2d43a"
format_stamp: "Formatted at 2026-08-09 19:30:02 on dist-test-slave-njxd"
server_key: "ef0e097e27d62f499b4523118ef008e0"
server_key_iv: "3a643c455be04e66254fa91f4cfee180"
server_key_version: "encryptionkey@0"
I20260809 19:30:02.398871 13516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:02.404765 13508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } attempt: 2
I20260809 19:30:02.405158 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.405018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:36645 (local address 127.0.0.1:36456)
0809 19:30:02.405081 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.405120 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:36645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 19:30:02.405514 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.405420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:36645 (local address 127.0.0.1:36468)
0809 19:30:02.405464 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.405492 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.253:36645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 19:30:02.405651 13508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.253:36645: Network error: Client connection negotiation failed: client connection to 127.21.81.253:36645: connect: Connection refused (error 111)
I20260809 19:30:02.410640 13516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:02.412910 13516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:02.413071 13516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:02.417315 13516 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.253:36645
I20260809 19:30:02.417361 13592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.253:36645 every 8 connection(s)
I20260809 19:30:02.417786 13516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/data/info.pb
I20260809 19:30:02.418777 13593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:02.420547 13593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:30:02.422863 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 13516
I20260809 19:30:02.422945 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-1/wal/instance
I20260809 19:30:02.423136 21831 external_mini_cluster.cc:1428] Setting key c52423540dfc0563b16f093ba4da22ca
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.253@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:30:02.430656 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.421265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36200 (local address 127.21.81.254:45875)
0809 19:30:02.421341 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.421343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.421502 (+ 159us) server_negotiation.cc:407] Connection header received
0809 19:30:02.422027 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.422028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.422040 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.422111 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.422691 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.423290 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.424712 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.424920 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.425023 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.427311 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:02.427313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:02.427314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:02.427322 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:02.429134 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.429566 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.429567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.429568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.429657 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.429937 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.429939 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.429939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.430096 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:02.430171 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:02.430433 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 19:30:02.430532 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 19:30:02.430682 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.421212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:45875 (local address 127.0.0.1:36200)
0809 19:30:02.421383 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.421420 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:02.421938 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:02.422163 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:02.422166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:02.422414 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:02.422612 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.422616 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.423383 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.423384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.424622 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.424626 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.425068 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.425070 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.425186 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.425775 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:02.425783 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:02.427210 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:02.429228 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.429235 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.429237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.429483 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.429749 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.429750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.429750 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.429864 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.430170 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:02.430175 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:02.430296 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 19:30:02.430383 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 19:30:02.430511 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":105,"threads_started":1}
I20260809 19:30:02.431473 13593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.81.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.81.252@KRBTEST.COM" created.
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.253@KRBTEST.COM for kudu/127.21.81.253@KRBTEST.COM
I20260809 19:30:02.439332 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.431763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:36645 (local address 127.0.0.1:36470)
0809 19:30:02.431987 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.431999 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:02.432044 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:02.432208 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:02.432209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:02.432372 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:02.432441 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.432443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.433258 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.433259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.433832 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.433834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.434290 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.434291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.434345 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.434732 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:02.434736 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:02.436058 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:02.437999 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.438001 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.438001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.438275 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.438482 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.438483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.438483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.438555 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.438875 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:02.438876 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:02.438930 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 19:30:02.439000 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 19:30:02.439119 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207}
I20260809 19:30:02.439369 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.431841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36470 (local address 127.21.81.253:36645)
0809 19:30:02.432044 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.432047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.432093 (+ 46us) server_negotiation.cc:407] Connection header received
0809 19:30:02.432112 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.432114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.432125 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.432185 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.432510 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.433176 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.433901 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.434201 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.434259 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.436143 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:02.436152 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:02.436153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:02.436160 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:02.437908 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.438358 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.438361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.438361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.438416 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.438613 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.438614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.438614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.438788 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:02.438883 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:02.439064 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 19:30:02.439230 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":161,"thread_start_us":89,"threads_started":1}
I20260809 19:30:02.440039 13593 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:30:02.440415 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.440324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:36663 (local address 127.0.0.1:45346)
0809 19:30:02.440357 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.440399 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260809 19:30:02.440579 13538 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
I20260809 19:30:02.441020 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.440931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:36663 (local address 127.0.0.1:45352)
0809 19:30:02.440979 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.441004 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 19:30:02.441159 13593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.252:36663: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.81.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.252.keytab.
Entry for principal kudu/127.21.81.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.252.keytab.
I20260809 19:30:02.450721 21831 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkvvQ95/build/release/bin/kudu
/tmp/dist-test-taskkvvQ95/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.252:36663
--webserver_interface=127.21.81.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.252.keytab
--principal=kudu/127.21.81.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.81.212:46705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.81.254:45875,127.21.81.253:36645,127.21.81.252:36663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:30:02.490427 13593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } attempt: 1
I20260809 19:30:02.490998 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.490853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:36663 (local address 127.0.0.1:45360)
0809 19:30:02.490921 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.490966 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 19:30:02.491387 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.491299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:36663 (local address 127.0.0.1:45362)
0809 19:30:02.491343 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.491370 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 19:30:02.491492 13593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.252:36663: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
I20260809 19:30:02.496821 13508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } attempt: 3
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for kudu/127.21.81.253@KRBTEST.COM
I20260809 19:30:02.504299 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.497173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:36645 (local address 127.0.0.1:36478)
0809 19:30:02.497257 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.497271 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:02.497372 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:02.497530 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:02.497531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:02.497740 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:02.497806 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.497808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.498454 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.498456 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.498994 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.498995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.499386 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.499387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.499475 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.500025 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:02.500030 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:02.501422 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:02.503227 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.503228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.503228 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.503406 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.503620 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.503621 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.503622 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.503689 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.503953 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:02.503954 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:02.504003 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 19:30:02.504094 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 19:30:02.504198 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 19:30:02.504299 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.497321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36478 (local address 127.21.81.253:36645)
0809 19:30:02.497389 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.497391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.497402 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:30:02.497432 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.497433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.497450 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.497509 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.497888 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.498367 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.499085 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.499310 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.499389 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.501517 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:02.501518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:02.501519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:02.501527 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:02.503119 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.503492 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.503493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.503493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.503552 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.503763 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.503764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.503764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.503886 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:02.503961 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:02.504120 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 19:30:02.504199 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 19:30:02.504978 13508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:30:02.505370 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.505248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:36663 (local address 127.0.0.1:45372)
0809 19:30:02.505302 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.505336 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 19:30:02.505674 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.505609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:36663 (local address 127.0.0.1:45384)
0809 19:30:02.505635 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.505655 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260809 19:30:02.505777 13508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.252:36663: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
W20260809 19:30:02.535005 13602 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:30:02.535270 13602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:30:02.535317 13602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:30:02.535356 13602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:30:02.536497 13602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:30:02.536561 13602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:30:02.536584 13602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:30:02.536607 13602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:30:02.536634 13602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:30:02.536899 13508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } attempt: 1
I20260809 19:30:02.537333 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.537189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:36663 (local address 127.0.0.1:45396)
0809 19:30:02.537254 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.537290 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 19:30:02.537665 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.537560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:36663 (local address 127.0.0.1:45404)
0809 19:30:02.537617 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.537641 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 19:30:02.537756 13508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.252:36663: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
I20260809 19:30:02.538445 13602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.81.212:46705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.81.254:45875,127.21.81.253:36645,127.21.81.252:36663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/krb5kdc/kudu_127.21.81.252.keytab
--principal=kudu/127.21.81.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.81.252:36663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.81.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:30:02.538743 13602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:30:02.538978 13602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:30:02.543716 13608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:02.543695 13611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:30:02.543695 13609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:30:02.543902 13602 server_base.cc:1034] running on GCE node
I20260809 19:30:02.544276 13602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:30:02.544580 13602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:30:02.545718 13602 hybrid_clock.cc:648] HybridClock initialized: now 1786303802545698 us; error 40 us; skew 500 ppm
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:30:02.548242 13602 init.cc:377] Logged in from keytab as kudu/127.21.81.252@KRBTEST.COM (short username kudu)
I20260809 19:30:02.548702 13602 webserver.cc:460] Webserver started at http://127.21.81.252:42913/ using document root <none> and password file <none>
I20260809 19:30:02.548939 13602 fs_manager.cc:359] Metadata directory not provided
I20260809 19:30:02.548986 13602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:30:02.549149 13602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:30:02.550258 13602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/data/instance:
uuid: "8a5ece7605c9446abb7792be5da528f8"
format_stamp: "Formatted at 2026-08-09 19:30:02 on dist-test-slave-njxd"
server_key: "05eedc6b9ae1d4f04ff6f3b26e2d7fa2"
server_key_iv: "ee5a6e478b2065ed1521cfff3e96c367"
server_key_version: "encryptionkey@0"
I20260809 19:30:02.550625 13602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/wal/instance:
uuid: "8a5ece7605c9446abb7792be5da528f8"
format_stamp: "Formatted at 2026-08-09 19:30:02 on dist-test-slave-njxd"
server_key: "05eedc6b9ae1d4f04ff6f3b26e2d7fa2"
server_key_iv: "ee5a6e478b2065ed1521cfff3e96c367"
server_key_version: "encryptionkey@0"
I20260809 19:30:02.551975 13602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:30:02.552680 13620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:30:02.552876 13602 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:30:02.552968 13602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/data,/tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/wal
uuid: "8a5ece7605c9446abb7792be5da528f8"
format_stamp: "Formatted at 2026-08-09 19:30:02 on dist-test-slave-njxd"
server_key: "05eedc6b9ae1d4f04ff6f3b26e2d7fa2"
server_key_iv: "ee5a6e478b2065ed1521cfff3e96c367"
server_key_version: "encryptionkey@0"
I20260809 19:30:02.553030 13602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:30:02.559648 13593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } attempt: 2
I20260809 19:30:02.560134 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.560003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:36663 (local address 127.0.0.1:45412)
0809 19:30:02.560062 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.560102 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 19:30:02.560483 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.560402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:36663 (local address 127.0.0.1:45426)
0809 19:30:02.560444 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.560468 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 19:30:02.560601 13593 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.81.252:36663: Network error: Client connection negotiation failed: client connection to 127.21.81.252:36663: connect: Connection refused (error 111)
I20260809 19:30:02.566934 13602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:30:02.568838 13602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:30:02.568994 13602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:30:02.572890 13602 rpc_server.cc:307] RPC server started. Bound to: 127.21.81.252:36663
I20260809 19:30:02.572939 13680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.81.252:36663 every 8 connection(s)
I20260809 19:30:02.573298 13602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/data/info.pb
I20260809 19:30:02.574327 13682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:30:02.575832 13682 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:30:02.576196 21831 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkvvQ95/build/release/bin/kudu as pid 13602
I20260809 19:30:02.576319 21831 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkvvQ95/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303525464116-21831-0/minicluster-data/master-2/wal/instance
I20260809 19:30:02.576534 21831 external_mini_cluster.cc:1428] Setting key 2fc4f641b0cbfeda65dcd99844075588
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.252@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.252@KRBTEST.COM
I20260809 19:30:02.585208 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.576565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36212 (local address 127.21.81.254:45875)
0809 19:30:02.576631 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.576633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.576929 (+ 296us) server_negotiation.cc:407] Connection header received
0809 19:30:02.577449 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.577451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.577464 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.577519 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.578087 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.578569 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.579444 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.579640 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.579727 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.582228 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:02.582230 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:02.582230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:02.582236 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:02.583848 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.584222 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.584223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.584224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.584286 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.584522 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.584522 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.584523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.584665 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:02.584749 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:02.585034 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 19:30:02.585116 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:30:02.585541 13684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.576538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.254:45875 (local address 127.0.0.1:36212)
0809 19:30:02.576722 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.576774 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:02.577338 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:02.577537 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:02.577539 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:02.577837 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:02.577989 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.577993 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.578662 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.578663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.579350 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.579352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.579763 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.579765 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.579913 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.580416 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:02.580425 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:02.582141 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:02.583936 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.583943 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.583944 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.584150 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.584361 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.584361 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.584361 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.584456 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.584754 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:02.584762 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:02.584882 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 19:30:02.584981 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 19:30:02.585125 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":18,"thread_start_us":102,"threads_started":1}
I20260809 19:30:02.585893 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.576933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45436 (local address 127.21.81.252:36663)
0809 19:30:02.577057 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.577062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.577229 (+ 167us) server_negotiation.cc:407] Connection header received
0809 19:30:02.577355 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.577360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.577379 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.577470 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.578125 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.578904 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.579648 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.579977 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.580059 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.582978 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:02.582985 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:02.582985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:02.582991 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:02.584536 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.584861 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.584864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.584864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.584925 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.585122 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.585123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.585123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.585288 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:02.585434 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:02.585696 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 19:30:02.585792 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":94,"thread_start_us":68,"threads_started":1}
I20260809 19:30:02.586122 13682 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.252@KRBTEST.COM for kudu/127.21.81.253@KRBTEST.COM
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.254@KRBTEST.COM
I20260809 19:30:02.592444 13684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.586349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.253:36645 (local address 127.0.0.1:36484)
0809 19:30:02.586397 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.586409 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:02.586444 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:02.586569 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:02.586569 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:02.586674 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:02.586705 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.586706 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.587428 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.587429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.587905 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.587906 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.588303 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.588304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.588360 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.588757 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:02.588760 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:02.589829 (+ 1069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:02.591459 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.591460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.591460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.591623 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.591832 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.591832 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.591833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.591895 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.592141 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:02.592142 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:02.592183 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 19:30:02.592262 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 19:30:02.592355 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 19:30:02.592456 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.586416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36484 (local address 127.21.81.253:36645)
0809 19:30:02.586461 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.586462 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.586470 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:02.586495 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.586496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.586510 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.586567 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.586778 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.587349 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.587994 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.588218 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.588302 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.589928 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:02.589929 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:02.589930 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:02.589936 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:02.591367 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.591695 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.591696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.591697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.591758 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.591957 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.591958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.591958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.592066 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:02.592136 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:02.592301 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 19:30:02.592372 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:30:02.593060 13682 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:30:02.594235 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.586871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36218 (local address 127.21.81.254:45875)
0809 19:30:02.586932 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.586934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.586941 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:02.586960 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.586960 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.586970 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.587031 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.587443 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.587879 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.588800 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.588997 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.589072 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.591497 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:02.591498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:02.591498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:02.591505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:02.593073 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.593472 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.593473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.593474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.593526 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.593753 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.593755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.593755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.593863 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:02.594023 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:02.594062 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 19:30:02.594132 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.252@KRBTEST.COM for kudu/127.21.81.252@KRBTEST.COM
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.81.253@KRBTEST.COM
I20260809 19:30:02.599992 13684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.593296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:36663 (local address 127.0.0.1:45438)
0809 19:30:02.593345 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.593355 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:02.593395 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:02.593505 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:02.593506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:02.593666 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:02.593694 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.593696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.594283 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.594284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.594973 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.594976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.595353 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.595354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.595402 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.595798 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:02.595802 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:02.597046 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:02.598811 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.598812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.598812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.599014 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.599294 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.599295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.599295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.599354 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.599648 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:02.599649 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:02.599688 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 19:30:02.599769 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 19:30:02.599868 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 19:30:02.600037 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.593346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.21.81.252:36663)
0809 19:30:02.593381 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.593382 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.593391 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:30:02.593450 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.593451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.593459 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.593505 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.593770 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.594207 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.595050 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.595277 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.595356 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.597121 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:02.597122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:02.597122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:02.597129 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:02.598729 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.599139 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.599141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.599141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.599205 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.599437 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.599438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.599438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.599548 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:02.599654 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:02.599791 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 19:30:02.599916 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":19}
I20260809 19:30:02.601960 13682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8: Bootstrap starting.
I20260809 19:30:02.602214 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.594975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36496 (local address 127.21.81.253:36645)
0809 19:30:02.595019 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.595021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.595028 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:02.595049 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.595050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.595059 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.595107 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.595546 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.595982 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.596919 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.597139 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.597212 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.599486 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:02.599487 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:02.599488 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:02.599495 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:02.601096 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.601444 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.601445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.601445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.601499 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.601728 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.601729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.601729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.601864 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:02.602010 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:02.602050 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 19:30:02.602119 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:30:02.602954 13682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:02.603308 13682 log.cc:824] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:02.603614 21831 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:30:02.604143 13682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8: No bootstrap required, opened a new log
I20260809 19:30:02.604657 13479 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6ae6447c32684494be9023f73db5bc08"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36218
I20260809 19:30:02.606325 13682 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } }
I20260809 19:30:02.606479 13682 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a5ece7605c9446abb7792be5da528f8, State: Initialized, Role: FOLLOWER
I20260809 19:30:02.606567 13682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } }
I20260809 19:30:02.606870 13689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } } }
I20260809 19:30:02.606917 13682 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:02.606987 13689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:30:02.608777 13700 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:30:02.608846 13700 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:30:02.609098 13602 master_runner.cc:186] Error getting master registration for 127.21.81.254:45875: OK, instance_id { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" instance_seqno: 1786303802268294 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:30:02.611230 13602 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:30:02.612887 13508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } attempt: 2
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.254@KRBTEST.COM for kudu/127.21.81.252@KRBTEST.COM
I20260809 19:30:02.619993 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.613213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45454 (local address 127.21.81.252:36663)
0809 19:30:02.613263 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.613265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.613277 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:30:02.613346 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.613347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.613357 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.613413 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.613765 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.614225 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.614935 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.615150 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.615228 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.617177 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:02.617178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:02.617179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:02.617187 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:02.618830 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.619173 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.619175 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.619175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.619237 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.619442 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.619443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.619443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.619581 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:02.619654 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:02.619825 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 19:30:02.619900 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:30:02.619993 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.613187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:36663 (local address 127.0.0.1:45454)
0809 19:30:02.613233 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.613247 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:02.613295 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:02.613417 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:02.613418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:02.613634 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:02.613689 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.613690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.614327 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.614328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.614851 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.614852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.615228 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.615229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.615287 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.615791 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:02.615796 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:02.617088 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:02.618923 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.618924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.618924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.619099 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.619310 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.619311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.619312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.619378 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.619659 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:02.619660 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:02.619714 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 19:30:02.619799 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 19:30:02.619900 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 19:30:02.621321 13508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08: Bootstrap starting.
I20260809 19:30:02.622030 13508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:02.622327 13508 log.cc:824] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:02.623085 13508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08: No bootstrap required, opened a new log
I20260809 19:30:02.624090 13508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } }
I20260809 19:30:02.624224 13508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae6447c32684494be9023f73db5bc08, State: Initialized, Role: FOLLOWER
I20260809 19:30:02.624336 13508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } }
I20260809 19:30:02.624614 13703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } } }
I20260809 19:30:02.624675 13508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:02.624689 13703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:30:02.624770 13479 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 19:30:02.624922 13479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:02.625845 13479 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } }
W20260809 19:30:02.626744 13715 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:30:02.626853 13715 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:30:02.627022 13652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ae6447c32684494be9023f73db5bc08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8a5ece7605c9446abb7792be5da528f8"
I20260809 19:30:02.627167 13652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:02.628034 13652 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ae6447c32684494be9023f73db5bc08 in term 1.
I20260809 19:30:02.629839 13455 leader_election.cc:302] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ae6447c32684494be9023f73db5bc08, 8a5ece7605c9446abb7792be5da528f8; no voters:
I20260809 19:30:02.629968 13703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:30:02.630113 13703 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [term 1 LEADER]: Becoming Leader. State: Replica: 6ae6447c32684494be9023f73db5bc08, State: Running, Role: LEADER
I20260809 19:30:02.630193 13703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } }
I20260809 19:30:02.630599 13703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ae6447c32684494be9023f73db5bc08. Latest consensus state: current_term: 1 leader_uuid: "6ae6447c32684494be9023f73db5bc08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } } }
I20260809 19:30:02.630692 13703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:02.630991 13479 leader_election.cc:288] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [CANDIDATE]: Term 1 election: Requested vote from peers a9b433ca1896463f8edbcca120f2d43a (127.21.81.253:36645), 8a5ece7605c9446abb7792be5da528f8 (127.21.81.252:36663)
I20260809 19:30:02.628850 13433 master_runner.cc:186] Error getting master registration for 127.21.81.253:36645: OK, instance_id { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" instance_seqno: 1786303802416183 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:30:02.631162 13433 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:30:02.632598 13719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:30:02.632782 13719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:30:02.635217 13652 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8 [term 1 FOLLOWER]: Refusing update from remote peer 6ae6447c32684494be9023f73db5bc08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:02.635619 13703 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:30:02.636968 13689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ae6447c32684494be9023f73db5bc08. Latest consensus state: current_term: 1 leader_uuid: "6ae6447c32684494be9023f73db5bc08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } } }
I20260809 19:30:02.637060 13689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:30:02.638236 13728 mvcc.cc:205] Tried to move back new op lower bound from 7316700375590842368 to 7316700375573950464. Current Snapshot: MvccSnapshot[applied={T|T < 7316700375590842368}]
I20260809 19:30:02.639113 13689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ae6447c32684494be9023f73db5bc08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } } }
I20260809 19:30:02.639216 13689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:30:02.639290 13729 mvcc.cc:205] Tried to move back new op lower bound from 7316700375590842368 to 7316700375573950464. Current Snapshot: MvccSnapshot[applied={T|T < 7316700375590842368}]
I20260809 19:30:02.639683 13703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ae6447c32684494be9023f73db5bc08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } } }
I20260809 19:30:02.639760 13703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:02.643828 13719 catalog_manager.cc:1347] Generated new cluster ID: 9a7465a7c16a48d39904675e408581d2
I20260809 19:30:02.643880 13719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:30:02.651757 13593 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } attempt: 3
I20260809 19:30:02.654826 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.632415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36226 (local address 127.21.81.254:45875)
0809 19:30:02.632453 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.632455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.637731 (+ 5276us) server_negotiation.cc:407] Connection header received
0809 19:30:02.637875 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.637876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.637887 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.637951 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.639666 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.640420 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.641546 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.641883 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.641952 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.643570 (+ 1618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:02.643572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:02.643572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:02.643579 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:02.645693 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.653950 (+ 8257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.653951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.653952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.654012 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.654217 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.654218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.654218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.654344 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:02.654419 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:02.654650 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 19:30:02.654734 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:30:02.657176 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.632662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36504 (local address 127.21.81.253:36645)
0809 19:30:02.632728 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.632730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.645740 (+ 13010us) server_negotiation.cc:407] Connection header received
0809 19:30:02.645845 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.645847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.645860 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.645917 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.646483 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.647263 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.648874 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.649180 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.649243 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.651334 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:02.651335 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:02.651335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:02.651343 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:02.655500 (+ 4157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.655881 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.655883 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.655883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.655996 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.656207 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.656208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.656209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.656449 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:02.656539 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:02.656819 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 19:30:02.657060 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 19:30:02.657311 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.633198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45468 (local address 127.21.81.252:36663)
0809 19:30:02.633237 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.633239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.645927 (+ 12688us) server_negotiation.cc:407] Connection header received
0809 19:30:02.646572 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.646573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.646592 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.646622 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.647195 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.647894 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.650035 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.650335 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.650375 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.653294 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:02.653295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:02.653296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:02.653302 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:02.654967 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.655962 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.655963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.655963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.656016 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.656364 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.656365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.656366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.656511 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:02.656611 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:02.657110 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 19:30:02.657179 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 19:30:02 dist-test-slave-njxd krb5kdc[13418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.81.253@KRBTEST.COM for kudu/127.21.81.252@KRBTEST.COM
I20260809 19:30:02.665375 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.652002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.81.252:36663 (local address 127.0.0.1:45470)
0809 19:30:02.652095 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 19:30:02.652105 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:30:02.652142 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:30:02.657515 (+ 5373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:30:02.657517 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:30:02.657785 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:30:02.657826 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.657828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.658663 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.658664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.659454 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:30:02.659457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.660221 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.660223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:30:02.660281 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.660675 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:30:02.660680 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:30:02.662229 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:30:02.664208 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.664209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.664209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.664435 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.664688 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:30:02.664689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:30:02.664690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:30:02.664760 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:30:02.665064 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:30:02.665065 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:30:02.665131 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 19:30:02.665191 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:30:02.665280 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 19:30:02.666194 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:02.652215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45470 (local address 127.21.81.252:36663)
0809 19:30:02.657376 (+ 5161us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.657378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.657385 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:30:02.657402 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.657402 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.657411 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.657466 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.657914 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.658597 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.659797 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:02.660110 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:02.660172 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:02.662337 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:30:02.662339 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:30:02.662339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:30:02.662345 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:30:02.664095 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.664531 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.664533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.664533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.664603 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:30:02.664835 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:30:02.664836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:30:02.664837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:30:02.664967 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:30:02.665023 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:02.665854 (+ 831us) server_negotiation.cc:299] Negotiation successful
0809 19:30:02.665987 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5147,"thread_start_us":93,"threads_started":1}
I20260809 19:30:02.667940 13593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a: Bootstrap starting.
I20260809 19:30:02.668674 13593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a: Neither blocks nor log segments found. Creating new log.
I20260809 19:30:02.668969 13593 log.cc:824] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:30:02.669900 13593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a: No bootstrap required, opened a new log
I20260809 19:30:02.671037 13719 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:30:02.672436 13719 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:30:02.672928 13593 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } }
I20260809 19:30:02.673071 13593 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b433ca1896463f8edbcca120f2d43a, State: Initialized, Role: FOLLOWER
I20260809 19:30:02.673152 13593 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } }
I20260809 19:30:02.673601 13593 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:30:02.673655 13563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ae6447c32684494be9023f73db5bc08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a9b433ca1896463f8edbcca120f2d43a"
I20260809 19:30:02.673765 13563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:30:02.674647 13563 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ae6447c32684494be9023f73db5bc08 in term 1.
I20260809 19:30:02.674925 13562 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a [term 1 FOLLOWER]: Refusing update from remote peer 6ae6447c32684494be9023f73db5bc08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:30:02.675159 13703 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 19:30:02.678488 13756 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:30:02.678556 13756 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:30:02.678649 13754 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ae6447c32684494be9023f73db5bc08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } } }
I20260809 19:30:02.678726 13754 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:30:02.678785 13754 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ae6447c32684494be9023f73db5bc08. Latest consensus state: current_term: 1 leader_uuid: "6ae6447c32684494be9023f73db5bc08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } } }
I20260809 19:30:02.678833 13754 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:30:02.678872 13754 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ae6447c32684494be9023f73db5bc08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } } }
I20260809 19:30:02.678932 13754 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:30:02.679569 13752 mvcc.cc:205] Tried to move back new op lower bound from 7316700375733866496 to 7316700375573950464. Current Snapshot: MvccSnapshot[applied={T|T < 7316700375590842368}]
I20260809 19:30:02.683856 13703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ae6447c32684494be9023f73db5bc08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ae6447c32684494be9023f73db5bc08" member_type: VOTER last_known_addr { host: "127.21.81.254" port: 45875 } } peers { permanent_uuid: "a9b433ca1896463f8edbcca120f2d43a" member_type: VOTER last_known_addr { host: "127.21.81.253" port: 36645 } } peers { permanent_uuid: "8a5ece7605c9446abb7792be5da528f8" member_type: VOTER last_known_addr { host: "127.21.81.252" port: 36663 } } }
I20260809 19:30:02.683977 13703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08 [sys.catalog]: This master's current role is: LEADER
I20260809 19:30:02.688623 13719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ae6447c32684494be9023f73db5bc08: Generated new TSK 0
I20260809 19:30:02.688814 13719 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 19:30:02.713434 13599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:02.711813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36514 (local address 127.21.81.253:36645)
0809 19:30:02.711847 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.711849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.712527 (+ 678us) server_negotiation.cc:407] Connection header received
0809 19:30:02.712765 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.712766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.712778 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.712810 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.713370 (+ 560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36514: BlockingRecv error: recv got EOF from 127.0.0.1:36514 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 19:30:02.714144 13513 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:02.711591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36236 (local address 127.21.81.254:45875)
0809 19:30:02.711638 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.711640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.712446 (+ 806us) server_negotiation.cc:407] Connection header received
0809 19:30:02.712878 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.712879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.712891 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.712920 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.714087 (+ 1167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36236: BlockingRecv error: recv got EOF from 127.0.0.1:36236 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 19:30:02.714937 13685 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:02.711978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45476 (local address 127.21.81.252:36663)
0809 19:30:02.712012 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.712014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.712714 (+ 700us) server_negotiation.cc:407] Connection header received
0809 19:30:02.714464 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.714465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.714476 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.714504 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.714881 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45476: BlockingRecv error: recv got EOF from 127.0.0.1:45476 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 19:30:02.717384 13599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:02.716671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36530 (local address 127.21.81.253:36645)
0809 19:30:02.716706 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.716707 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.716892 (+ 185us) server_negotiation.cc:407] Connection header received
0809 19:30:02.716992 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.716993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.717009 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.717036 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.717339 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36530: BlockingRecv error: recv got EOF from 127.0.0.1:36530 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 19:30:02.720618 13599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:02.719760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36532 (local address 127.21.81.253:36645)
0809 19:30:02.719813 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.719815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.720078 (+ 263us) server_negotiation.cc:407] Connection header received
0809 19:30:02.720194 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.720195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.720204 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.720233 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.720575 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36532: BlockingRecv error: recv got EOF from 127.0.0.1:36532 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 19:30:02.724617 13599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:02.723793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36546 (local address 127.21.81.253:36645)
0809 19:30:02.723829 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.723830 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.724104 (+ 274us) server_negotiation.cc:407] Connection header received
0809 19:30:02.724207 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.724208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.724218 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.724248 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.724574 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36546: BlockingRecv error: recv got EOF from 127.0.0.1:36546 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 19:30:02.730675 13599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:02.729857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36560 (local address 127.21.81.253:36645)
0809 19:30:02.729893 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.729895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.730164 (+ 269us) server_negotiation.cc:407] Connection header received
0809 19:30:02.730266 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.730267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.730277 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.730306 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.730628 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36560: BlockingRecv error: recv got EOF from 127.0.0.1:36560 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 19:30:02.740808 13599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:02.739910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36562 (local address 127.21.81.253:36645)
0809 19:30:02.739950 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.739952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.740247 (+ 295us) server_negotiation.cc:407] Connection header received
0809 19:30:02.740353 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.740354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.740365 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.740394 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.740753 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36562: BlockingRecv error: recv got EOF from 127.0.0.1:36562 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 19:30:02.759388 13599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:02.758389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36570 (local address 127.21.81.253:36645)
0809 19:30:02.758438 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.758441 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.758752 (+ 311us) server_negotiation.cc:407] Connection header received
0809 19:30:02.758864 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.758865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.758876 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.758910 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.759323 (+ 413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36570: BlockingRecv error: recv got EOF from 127.0.0.1:36570 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 19:30:02.793889 13599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:02.792925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36584 (local address 127.21.81.253:36645)
0809 19:30:02.792973 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.792976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.793280 (+ 304us) server_negotiation.cc:407] Connection header received
0809 19:30:02.793383 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.793384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.793396 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.793425 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.793832 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36584: BlockingRecv error: recv got EOF from 127.0.0.1:36584 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 19:30:02.860224 13599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:02.859509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36600 (local address 127.21.81.253:36645)
0809 19:30:02.859588 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.859590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.859600 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:30:02.859690 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.859691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.859703 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.859778 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.860154 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36600: BlockingRecv error: recv got EOF from 127.0.0.1:36600 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 19:30:02.990326 13599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:02.989518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36612 (local address 127.21.81.253:36645)
0809 19:30:02.989631 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:02.989633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:02.989743 (+ 110us) server_negotiation.cc:407] Connection header received
0809 19:30:02.989844 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:02.989845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:02.989857 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:02.989906 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:02.990261 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36612: BlockingRecv error: recv got EOF from 127.0.0.1:36612 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 19:30:03.248399 13599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:30:03.247673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36628 (local address 127.21.81.253:36645)
0809 19:30:03.247757 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:03.247759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:03.247771 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:30:03.247847 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:03.247848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:03.247859 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:03.247929 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:30:03.248327 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36628: BlockingRecv error: recv got EOF from 127.0.0.1:36628 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 19:30:03.609330 13700 catalog_manager.cc:1259] Loaded cluster ID: 9a7465a7c16a48d39904675e408581d2
I20260809 19:30:03.609411 13700 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8: loading cluster ID for follower catalog manager: success
I20260809 19:30:03.610184 13700 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8: acquiring CA information for follower catalog manager: success
I20260809 19:30:03.610457 13700 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8a5ece7605c9446abb7792be5da528f8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:30:03.679013 13756 catalog_manager.cc:1259] Loaded cluster ID: 9a7465a7c16a48d39904675e408581d2
I20260809 19:30:03.679095 13756 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a: loading cluster ID for follower catalog manager: success
I20260809 19:30:03.679888 13756 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a: acquiring CA information for follower catalog manager: success
I20260809 19:30:03.680136 13756 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a9b433ca1896463f8edbcca120f2d43a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:30:03.764679 13834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:03.761794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36638 (local address 127.21.81.253:36645)
0809 19:30:03.761935 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:03.761938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:03.762011 (+ 73us) server_negotiation.cc:407] Connection header received
0809 19:30:03.762118 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:03.762119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:03.762193 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:03.762285 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:03.762466 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:03.762982 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:03.763798 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:03.764016 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:03.764110 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:03.764309 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:03.764419 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:03.764510 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:03.764550 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 19:30:03.764576 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":58,"threads_started":1}
I20260809 19:30:03.768738 13840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:03.766596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45492 (local address 127.21.81.252:36663)
0809 19:30:03.766703 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:03.766704 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:03.766712 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:30:03.766748 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:03.766749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:03.766784 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:03.766849 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:03.766933 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:03.767397 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:03.768047 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:03.768227 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:03.768273 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:03.768399 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:03.768480 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:03.768550 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:03.768602 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 19:30:03.768619 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73,"thread_start_us":49,"threads_started":1}
I20260809 19:30:03.773016 13834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:03.770328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.21.81.253:36645)
0809 19:30:03.770463 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:03.770465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:03.770600 (+ 135us) server_negotiation.cc:407] Connection header received
0809 19:30:03.770644 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:03.770645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:03.770657 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:03.770710 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:03.770848 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:03.771359 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:03.772285 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:03.772538 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:03.772594 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:03.772725 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:03.772756 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:03.772805 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:03.772889 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 19:30:03.772911 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260809 19:30:03.773339 13840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:03.770354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45498 (local address 127.21.81.252:36663)
0809 19:30:03.770394 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:03.770395 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:03.770729 (+ 334us) server_negotiation.cc:407] Connection header received
0809 19:30:03.770826 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:03.770827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:03.770838 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:03.770868 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:03.771022 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:03.771685 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:03.772453 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:03.772726 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:03.772763 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:03.773012 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:03.773044 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:03.773105 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:03.773232 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 19:30:03.773274 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:30:03.773813 13847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:30:03.770373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36248 (local address 127.21.81.254:45875)
0809 19:30:03.770471 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 19:30:03.770472 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:30:03.770600 (+ 128us) server_negotiation.cc:407] Connection header received
0809 19:30:03.770654 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:30:03.770655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:30:03.770713 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:30:03.770786 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:30:03.770895 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:03.771643 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:03.772530 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:30:03.772837 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:30:03.772916 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:30:03.773242 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:30:03.773349 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:30:03.773411 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 19:30:03.773689 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 19:30:03.773721 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":50,"threads_started":1}
I20260809 19:30:03.775678 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 13433
I20260809 19:30:03.783856 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 13516
I20260809 19:30:03.790315 21831 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkvvQ95/build/release/bin/kudu with pid 13602
2026-08-09T19:30:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2842 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5680 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (278333 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST